Papers by Wang Xi

115 papers
UniCodec: Unified Audio Codec with Single Domain-Adaptive Codebook (2025.acl-long)

Copied to clipboard

Challenge: Existing neural audio codecs are not capable of handling multi-domain audio data . et al., 2023) integrate speech modality with text-based large language models .
Approach: They propose a unified audio codec with a single codebook to support multi-domain audio data . they propose combining a mix-of-experts strategy and a partitioned domain-adaptive codebook method .
Outcome: The proposed codec outperforms existing codecs on acoustic and semantic representation capabilities.
To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Models (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing unlearning paradigms are mired in vague forgetting boundaries, erasing knowledge indiscriminately.
Approach: They propose a benchmark to evaluate if unlearning erases essential knowledge . they propose 'knowUnDo' which uses copyrighted content and privacy domains .
Outcome: The proposed method is superior to existing methods in both precise knowledge unlearning and general knowledge retaining of LLMs.
Improving In-Context Learning with Prediction Feedback for Sentiment Analysis (2024.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) have achieved promising results in sentiment analysis through the in-context learning paradigm.
Approach: They propose a framework that incorporates prior predictions and feedback to improve sentiment understanding by incorporating prior feedback and leveraging a feedback-driven prompt.
Outcome: The proposed framework improves on nine sentiment analysis datasets with an average improvement of 5.95% over conventional methods.
OpenForecast: A Large-Scale Open-Ended Event Forecasting Dataset (2025.coling-main)

Copied to clipboard

Challenge: Existing closed-ended event forecasting methods are constrained by a limited answer space.
Approach: They introduce OpenForecast, a large-scale open-ended dataset with three open-ending event forecasting tasks and an automatic LLM-based method for complex events.
Outcome: The proposed method can be used to evaluate the ability of complex event forecasting of large language models.
A Survey on Asking Clarification Questions Datasets in Conversational Systems (2023.acl-long)

Copied to clipboard

Challenge: Existing studies on Asking Clarification Questions (ACQs) are incomparable due to inconsistent data, experimental setups and evaluation strategies.
Approach: They analyse the current research status on Asking Clarification Questions (ACQs) and propose a set of evaluation metrics and benchmarks for multiple ACQs-related tasks.
Outcome: The proposed techniques are compared with the available datasets and evaluated against benchmarks.
Exploring the Impact of Personality Traits on LLM Toxicity and Bias (2025.emnlp-main)

Copied to clipboard

Challenge: anthropomorphic LLMs are being developed to serve diversified roles, but content safety concerns remain regarding their toxicity and toxicity.
Approach: They propose to assign personality traits to large language models (LLMs) to reduce toxic language and social biases in their outputs by using the widely accepted HEXACO personality framework developed in social psychology.
Outcome: The proposed model is able to perform on three toxic and bias benchmarks and shows that assigning personality traits reduces bias and toxicity similar to humans’ correlations between personality traits and toxic behaviors.
Capturing Event Argument Interaction via A Bi-Directional Entity-Level Recurrent Decoder (2021.acl-long)

Copied to clipboard

Challenge: Existing efforts to capture event argument interactions are limited by the argument role type information of contextual entities.
Approach: They propose to capture event argument interactions as a Seq2Seq-like learning problem where a sentence with a specific event trigger is mapped to a sequence of event argument roles.
Outcome: The proposed neural architecture generates argument roles by incorporating contextual entities’ argument role predictions, like a word-by-word text generation process, thereby distinguishing implicit argument distribution patterns within an event more accurately.
Beyond Token Length: Step Pruner for Efficient and Accurate Reasoning in Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Existing reinforcement learning methods for large reasoning models suffer from excessive verbosity, known as "overthinking." Existing models penalize generated tokens to promote conciseness, but these methods encounter two challenges: they may develop hacking behavior in later stages of training by discarding reasoning steps.
Approach: They propose a framework that steers large reasoning models toward more efficient reasoning . they prioritize correctness while imposing penalties for redundant steps .
Outcome: The proposed framework reduces token usage by 69.7% on AIME24.
AS-ES Learning: Towards efficient CoT learning in small models (2024.findings-acl)

Copied to clipboard

Challenge: Existing methods to induce Chain-of-Thought (CoT) in LLMs are limited and do not consider the importance of efficiently utilizing existing CoT data.
Approach: They propose a new training paradigm which exploits the inherent information in CoT for iterative generation.
Outcome: The proposed training paradigm surpasses direct seq2seq training on CoT-extensive tasks without data augmentation or altering the model itself.
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified.
Approach: They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .
Outcome: Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 .
Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language Models (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) have demonstrated impressive capabilities across various domains, but are vulnerable to backdoor attacks.
Approach: They propose a chain-of-scrutiny approach which leverages LLMs’ unique reasoning abilities to mitigate backdoor attacks.
Outcome: The proposed model is well-suited for the popular API-only LLM deployments, enabling detection at minimal cost and with little data.
LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin (2024.acl-long)

Copied to clipboard

Challenge: Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM.
Approach: They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks.
Outcome: The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting.
Learning from Contrasts: Synthesizing Reasoning Paths from Diverse Search Trajectories (2026.acl-long)

Copied to clipboard

Challenge: MCTS methods retain only the single highest-reward trajectory, discarding comparative signals present in the many explored paths.
Approach: They propose a framework that transforms supervision extraction into a synthesis procedure.
Outcome: The proposed framework matches or exceeds baselines on 60K CRPS-synthesized examples on out-of-domain benchmarks.
Enhancing LLM Knowledge Learning through Generalization (2025.findings-emnlp)

Copied to clipboard

Challenge: Continued pre-training on paraphrased data has shown empirical promise for enhancing knowledge acquisition, but this approach is costly and unreliable as it relies on external models or manual effort for rewriting.
Approach: They propose formatting-based data augmentation which diversifies documents conveying the same knowledge by altering document formats rather than their content.
Outcome: The proposed methods improve generalization to diverse paraphrased contexts and enhance pre-training and instruction tuning.
MADial-Bench: Towards Real-world Evaluation of Memory-Augmented Dialogue Generation (2025.naacl-long)

Copied to clipboard

Challenge: Existing evaluation metrics for memory-augmented dialogue systems lack practical value . current evaluation methods only consider passive memory retrieval while ignoring diverse memory recall with rich triggering factors.
Approach: They propose to use long-term memory to create human-like dialogues using chatbots.
Outcome: The proposed benchmark covers memory retrieval and memory recognition tasks with both passive and proactive memory recall data.
Improving Conversational Recommendation Systems via Bias Analysis and Language-Model-Enhanced Data Augmentation (2023.findings-emnlp)

Copied to clipboard

Challenge: Conversational Recommendation System (CRS) is a rapidly growing research area, along with advancements in language modelling techniques.
Approach: They propose to use a benchmark dataset to develop CRS models and address biases arising from feedback loop inherent in multi-turn interactions to enhance model performance while mitigating biase.
Outcome: The proposed strategies improve on ReDial and TG-ReDial benchmark datasets and offer additional insights on addressing multiple newly formulated biases.
Multiple Knowledge-Enhanced Interactive Graph Network for Multimodal Conversational Emotion Recognition (2024.findings-emnlp)

Copied to clipboard

Challenge: Multimodal Emotion Recognition in Conversations models struggle due to lack of Common Sense Knowledge (CSK).
Approach: They propose a multimodal approach to integrate multiple knowledge into the edge representations by integrating textual and visual CSK.
Outcome: The proposed model outperforms state-of-the-art methods on two popular datasets.
Improving Discriminative Capability of Reward Models in RLHF Using Contrastive Learning (2024.emnlp-main)

Copied to clipboard

Challenge: Current methods rely on ranking losses to teach reward model to assess preferences, but they are susceptible to noise and ambiguous data, often failing to deeply understand human intentions.
Approach: They propose a method that incorporates contrastive learning into the reward modeling process to enhance generalization and stabilize the reinforcement learning training process.
Outcome: The proposed method enhances generalization of the reward model, stabilizes the reinforcement learning training process, and improves the final alignment with human preferences.
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored.
Approach: They propose a survey structured around the pipeline to identify and improve MI models.
Outcome: The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency.
Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks (N19-1)

Copied to clipboard

Challenge: Existing distance supervised relation extraction models for long-tail data are inadequate for many applications.
Approach: They propose to leverage implicit relational knowledge among class labels and learn explicit relational knowing using graph convolution networks.
Outcome: The proposed approach outperforms baselines for long-tail relations on a large-scale dataset.
UniCoRN: Unified Cognitive Signal ReconstructioN bridging cognitive signals and human language (2023.acl-long)

Copied to clipboard

Challenge: Existing studies focus on decoding word-level fMRI volumes from a restricted vocabulary.
Approach: They propose an open-vocabulary task to bridge fMRI time series and human language . they use a pre-trained language model to construct a robust encoder for cognitive signals .
Outcome: The proposed task bridges fMRI time series and human language with a baseline model.
Analyzing Chain-of-thought Prompting in Black-Box Large Language Models via Estimated V-information (2024.lrec-main)

Copied to clipboard

Challenge: Chain-of-Thought (CoT) prompting and large language models (LLMs) have shown great potential in improving performance on challenging reasoning tasks.
Approach: They propose a new metric which extends the concept of pointwise V-information to black-box models and quantifies label-relevant new information introduced by CoT prompting.
Outcome: The proposed metric extends the concept of pointwise V-information to black-box models, quantifying label-relevant new information introduced by CoT prompting beyond pre-existing label information.
E-ConvRec: A Large-Scale Conversational Recommendation Dataset for E-Commerce Customer Service (2022.lrec-1)

Copied to clipboard

Challenge: Recent research has focused on developing conversational recommendation system (CRS), which provides valuable recommendations to users through conversations.
Approach: They construct an authentic Chinese dialogue dataset consisting of over 25k dialogues and 770k utterances, which contains user profile, product knowledge base, and multiple sequential real conversations between users and recommenders.
Outcome: The proposed dataset contains user profile, product knowledge base, and multiple sequential real conversations between users and recommenders.
Toward Better Loanword Identification in Uyghur Using Cross-lingual Word Embeddings (C18-1)

Copied to clipboard

Challenge: Almost every natural language processing task suffers from data sparseness.
Approach: They propose a method which identify loanwords in monolingual corpora by using cross-lingual word embeddings as core feature and a log-linear model which combines several shallow features to predict the final results.
Outcome: The proposed method outperforms baseline models significantly on loanword identification and translation in four languages and eight translation directions.
SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis (2025.findings-naacl)

Copied to clipboard

Challenge: Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis.
Approach: They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis.
Outcome: SciAssess evaluates 11 LLMs on multiple tasks across scientific fields.
PFDial: A Structured Dialogue Instruction Fine-tuning Method Based on UML Flowcharts (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) have shown remarkable progress in dialogue and reasoning, but they struggle to solve strictly constrained dialogue tasks.
Approach: They construct a dataset that contains 12,705 high-quality Chinese dialogue instructions from 440 flowcharts containing 5,055 process nodes.
Outcome: The proposed model outperforms GPT-4o models on backward transitions and outperformed GPT-42 models on the same dataset.
ChildMandarin: A Comprehensive Mandarin Speech Dataset for Young Children Aged 3-5 (2025.acl-long)

Copied to clipboard

Challenge: Automatic speech recognition systems have advanced significantly with models like Whisper, Conformer, and self-supervised frameworks such as Wav2vec 2.0.
Approach: They propose to use Mandarin speech datasets to analyze pronunciation and tone of children aged 3 to 5 and evaluate their models on speaker verification (SV) They find that the datasets are more robust than those used by adult speech recognition systems and are open-source and available for all academic purposes.
Outcome: The proposed dataset includes 41.25 hours of speech with carefully crafted manual transcriptions, collected from 397 speakers across various provinces in China, with balanced gender representation.
Time-for-Accuracy: Formalizing Chain-of-Thought as an Expansion of Logical Depth (2026.findings-acl)

Copied to clipboard

Challenge: Chain-of-thought (CoT) prompting can improve multi-step reasoning, but it is unclear what kind of additional sequential computation longer traces actually enable.
Approach: They propose a deletion-based measure of step necessity under a specified inference interface to operationalize realized depth beyond raw length.
Outcome: The proposed method combines effective logical depth with Bennett's logical depth to show that it is more efficient than a linear model.
AgentGym: Evaluating and Training Large Language Model-based Agents across Diverse Environments (2025.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) are promising foundations to build generally-capable agents . however, the community lacks a unified interactive framework that covers diverse environments for comprehensive evaluation of agents.
Approach: They propose a framework that features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction.
Outcome: The proposed framework features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction.
ChatHLS: Towards Systematic Design Automation and Optimization for High-Level Synthesis (2026.acl-long)

Copied to clipboard

Challenge: High-Level Synthesis (HLS) is a hardware design tool that can be used to design hardware from C-like languages, but its widespread adoption is limited by strict coding constraints and design-specific optimizations.
Approach: They propose a multi-agent HLS design framework that leverages specialized LLMs for automated debugging and directive tuning.
Outcome: The proposed framework outperforms Gemini-3-pro in debugging and speedups across various HLS kernels and neural network accelerators.
Multi-Programming Language Sandbox for LLMs (2025.acl-demo)

Copied to clipboard

Challenge: MPLSandbox is an out-of-the-box multi-programming language sandbox designed to provide unified and comprehensive feedback from compiler and analysis tools for Large Language Models (LLMs).
Approach: They propose a multi-programming language sandbox that provides unified feedback from compilers and analysis tools for Large Language Models.
Outcome: The proposed multi-language sandbox can provide comprehensive feedback from compilers and analysis tools for large language models (LLMs).
Detoxifying Large Language Models via Knowledge Editing (2024.acl-long)

Copied to clipboard

Challenge: Existing methods to detoxify Large Language Models (LLMs) are limiting, but knowledge editing can be effective.
Approach: They propose a baseline method to detoxify Large Language Models (LLMs) they propose supervised fine-tuning and reinforcement learning from human feedback (RLHF)
Outcome: The proposed method reduces toxicity of large language models with one instance of tuning . it reduces the toxicity, while minimizing the toxins, the authors show .
SudokuFill: A Multi-Agent Progressive Filling Framework for Document-Level Scientific Information Extraction (2026.findings-acl)

Copied to clipboard

Challenge: Scientific information extraction (SciIE) is a key bottleneck for turning unstructured papers into computable knowledge bases.
Approach: They propose a scientific information extraction framework that solves a Sudoku problem as a progressive filling problem.
Outcome: The proposed framework outperforms the GPT-4o model on a document-level adjuvant dataset.
STELLA: A Multimodal LLM for Protein Functional Annotation via Unified Sequence-Structure Encoding (2026.findings-acl)

Copied to clipboard

Challenge: a multimodal protein language model (LLM) integrates sequence, structure, and function into functional annotation.
Approach: They propose a multimodal protein language model that synergistically aligns bimodal representations with the textual modality to advance protein functional annotation.
Outcome: The proposed model synergizes bimodal representations with the textual modality to advance protein functional annotation.
Clarifying the Path to User Satisfaction: An Investigation into Clarification Usefulness (2024.findings-eacl)

Copied to clipboard

Challenge: Poorly formulated questions can lead to user frustration and dissatisfaction .
Approach: They propose to leverage key features that contribute to the classification of clarifying questions, enhancing user satisfaction and system performance.
Outcome: The proposed model improves with a minimum performance boost of 45% in traditional classifiers, especially in large language models.
LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Models (2026.acl-long)

Copied to clipboard

Challenge: Existing evaluation of Large Language Models on static benchmarks is vulnerable to data contamination and leaderboard overfitting.
Approach: LLMEval-Fair framework provides a framework for dynamic evaluation of Large Language Models . evaluators use a proprietary bank of 220k graduate-level questions to analyze model data .
Outcome: LLMEval-Fair provides robust and credible evaluation framework for Large Language Models . it provides a strong empirical validation for the dynamic evaluation paradigm .
Improving Embedding-based Large-scale Retrieval via Label Enhancement (2021.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for large-scale retrieval are trained with 0-1 hard labels that indicate whether a query is relevant to a document, ignoring rich information of the relevance degree.
Approach: They propose to introduce label enhancement for the first time to characterize query-document relevance degree by embedding label distribution into contextual embeddables.
Outcome: The proposed method significantly outperforms existing retrieval models and its counterparts equipped with two alternative methods on English and Chinese large-scale retrieval tasks.
Efficient Self-Evaluation for Diffusion Language Models via Sequence Regeneration (2026.acl-long)

Copied to clipboard

Challenge: Non-sequential and bidirectional nature of diffusion large language models makes direct likelihood-based self-evaluation challenging.
Approach: They propose a self-evaluation confidence quantification method for diffusion large language models that quantifies confidence by computing the probability of regenerating tokens in the entire generated sequence, given the full context.
Outcome: The proposed method is correlated with semantic coherence and answer accuracy.
Sensitivity-LoRA : Low-Load Sensitivity-Based Fine-Tuning for Large Language Models (2025.findings-emnlp)

Copied to clipboard

Challenge: Low-Rank Adaptation (LoRA) is a promising approach to adapting LLMs to specialized tasks . existing rank allocation techniques remain computationally inefficient and unstable .
Approach: They propose a low-rank adapted model that approximates model weight updates using low-ranked decomposition.
Outcome: The proposed method is limited by its uniform rank allocation to each incremental matrix . it leverages the second-order derivatives of the loss function to capture weight sensitivity .
iAgent: LLM Agent as a Shield between User and Recommender Systems (2025.findings-acl)

Copied to clipboard

Challenge: Traditional recommender systems focus on the user-platform paradigm, where users are directly exposed under the control of the platform's recommendation algorithms.
Approach: They propose a user-agent-platform paradigm where agent serves as the protective shield between user and recommender system that enables indirect exposure.
Outcome: The proposed model improves 16.6% over baselines on four datasets and mitigates echo chamber effects and reduces model bias in disadvantaged users.
Can We Edit Multimodal Large Language Models? (2023.emnlp-main)

Copied to clipboard

Challenge: Existing methods to edit multimodal models have been used to incrementally infuse a language model with a new set of facts.
Approach: They construct a benchmark for editing multimodal Large Language Models and establish metrics for evaluation.
Outcome: The proposed benchmarks show that editing multimodal models is not as difficult as editing single-modal models.
Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods for parameter pruning fail to utilize the knowledge from pruned parameters.
Approach: They propose a method that uses manifold learning and the Information Bottleneck measure to merge similar layers to preserve model performance.
Outcome: The proposed method outperforms pruning methods on multiple datasets and LLMs with quantization and achieves substantial compression ratios.
Low-Resource Language Expansion and Translation Capacity Enhancement for LLM: A Study on the Uyghur (2025.coling-main)

Copied to clipboard

Challenge: Extensive experiments have shown that our strategy effectively expands the low-resource languages supported by large language models and significantly enhances the model’s translation ability in Uyghur with less parallel data.
Approach: They propose a direct preference optimization based on translation self-evolution to expand low-resource languages into large language models by using Uyghur as an example.
Outcome: The proposed strategy expands low-resource languages supported by large language models and significantly enhances the model’s translation ability in Uyghur with less parallel data.
RIPRAG: Hack a Black-box Retrieval-Augmented Generation Question-Answering System with Reinforcement Learning (2026.findings-acl)

Copied to clipboard

Challenge: Existing methods to generate RAG documents require knowledge of the target RAG system’s internal composition and implementation details, whereas black-box methods are unable to utilize interactive information.
Approach: They propose a RIPRAG attack framework that treats the target RAG system as a black box and leverages a Reinforcement Learning from Black-box Feedback (RLBF) method to optimize the generation model for poisoned documents.
Outcome: The proposed method achieves an attack success rate (ASR) improvement of up to 0.72 compared to baseline methods.
HyperAdaLoRA: Accelerating LoRA Rank Allocation During Training via Hypernetworks without Sacrificing Performance (2026.findings-acl)

Copied to clipboard

Challenge: Low-Rank Adaptation (LoRA) assumes a uniform rank r for each incremental matrix, not accounting for the varying significance of weight matrices across modules and layers.
Approach: They propose a framework that allows for faster convergence of low-rank adaptive models . they use a hypernetwork to prune the outputs of the hypernetworks to generate parameters .
Outcome: The proposed framework accelerates convergence of AdaLoRA by leveraging a hypernetwork.
PiKGL: Leveraging Pruned Knowledge Graphs for Explainable Stance Detection (2026.tacl-1)

Copied to clipboard

Challenge: Experimental results demonstrate that a Pruned interpretable knowledge Graph Learning framework for explainable stance detection is state-of-the-art for social media stance prediction.
Approach: They propose a Pruned interpretable knowledge Graph Learning framework for explainable stance detection that incorporates commonsense knowledge and prunes redundant information to ensure precision and minimize noise.
Outcome: The proposed framework achieves state-of-the-art on three public datasets.
Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling (2025.findings-emnlp)

Copied to clipboard

Challenge: a framework for constructing dialogue world models for natural language tasks is currently lacking.
Approach: They propose a framework that can be used to train a dialogue world model.
Outcome: The proposed framework can predict future utterances and user beliefs . it can achieve state-of-the-art performance on emotion classification and sentiment identification .
Finding Influential Instances for Distantly Supervised Relation Extraction (2022.coling-1)

Copied to clipboard

Challenge: Distant supervision models suffer from high label noise and are not reliable for DS.
Approach: They propose a model-agnostic instance sampling method for relation extraction (RE) by influence function, namely REIF.
Outcome: The proposed method reduces the computational complexity from O(mn) to O(1), with analyzing its robustness on the selected sampling function.
MetaASSIST: Robust Dialogue State Tracking with Meta Learning (2022.emnlp-main)

Copied to clipboard

Challenge: Existing dialogue datasets contain lots of noise in their state annotations.
Approach: They propose a framework to train robust dialogue state tracking models by combining pseudo and vanilla labels by a common weighting parameter.
Outcome: The proposed framework achieves state-of-the-art accuracy of 80.10% on multiWOZ 2.4.
Fundamental Reasoning Paradigms Induce Out-of-Domain Generalization in Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Deduction, induction, and abduction are fundamental reasoning paradigms, core for human logical thinking.
Approach: They propose to use a dataset of symbolic tasks to induce deductive skills into large language models (LLMs) they then use FT to fine-tune models to improve OOD generalization .
Outcome: The proposed approach yields strong generalizability with substantial performance gains (up to 14.60) across realistic out-of-domain tasks.
Reward Modeling Requires Automatic Adjustment Based on Data Quality (2024.findings-emnlp)

Copied to clipboard

Challenge: Reinforcement Learning from Human Feedback (RLHF) is a method for aligning language models with human values.
Approach: They propose a method that automatically adjusts reward modeling based on data quality . they use preference data to train a reward model that is more aligned with human values .
Outcome: The proposed method stabilizes reward model training and significantly improves alignment performance on human preference datasets.
Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptation (2026.acl-long)

Copied to clipboard

Challenge: Low-Rank Adaptation (LoRA) is a key parameter-efficient fine-tuning method . however, its effectiveness is hampered by semantic drift and structural incoherence .
Approach: They propose a low-rank Adaptation framework that tackles semantic drift and structural incoherence by pruning task-irrelevant directions.
Outcome: Experiments on large language models, vision models, and vision models show that the proposed framework outperforms LoRA and advanced dynamic rank allocation and sparsity-based methods.
Can MLLMs Understand the Deep Implication Behind Chinese Images? (2025.acl-long)

Copied to clipboard

Challenge: MLLMs perform poorly on traditional culture images, indicating limitations in understanding high-level semantics and lacking a deep knowledge base of Chinese traditional culture.
Approach: They propose to use Chinese images to assess MLLMs' higher-order perception and understanding of Chinese visual content.
Outcome: The proposed model incorporates images that represent Chinese traditional culture, such as famous Chinese traditional paintings, to ensure the authenticity of the Chinese context.
SMASH: Improving SMAll Language Models’ Few-SHot Ability with Prompt-Based Distillation (2022.findings-emnlp)

Copied to clipboard

Challenge: Large-scale language models with prompts have shown remarkable performance on few-shot learning.
Approach: They propose an approach to improve SMAll language models’ few-SHot ability by training on intermediate tasks before prompt-based fine-tuning on downstream tasks.
Outcome: The proposed model improves on sentence-pair and sentiment classification tasks by training on intermediate tasks before fine-tuning on downstream tasks.
PromptFix: Few-shot Backdoor Removal via Adversarial Prompt Tuning (2024.naacl-long)

Copied to clipboard

Challenge: Existing studies have shown that pre-trained language models can be backdoored such that model behavior is manipulated when trigger tokens are presented.
Approach: They propose a backdoor mitigation strategy for NLP models via adversarial prompt-tuning in few-shot settings that uses two extra sets of soft tokens which approximate the trigger and counteract it respectively.
Outcome: The proposed method keeps model parameters intact and only utilizes two extra sets of soft tokens which approximate the trigger and counteract it respectively.
WebWalker: Benchmarking LLMs in Web Traversal (2025.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of natural language processing tasks.
Approach: They propose a benchmark to assess the ability of LLMs to perform web traversal by using an explore-critic paradigm.
Outcome: The proposed framework mimics human-like web navigation through an explore-critic paradigm and demonstrates the effectiveness of RAG combined with WebWalker in real-world scenarios.
Label-Free Distant Supervision for Relation Extraction via Knowledge Graph Embedding (D18-1)

Copied to clipboard

Challenge: Existing methods to generate large scale labeled data for relation extraction produce noisy relation labels when there are multiple relationships between entities.
Approach: They propose a method which assumes that a pair of entities appears in a Knowledge Graph and trains a relation classifier.
Outcome: The proposed method performs well in the current distant supervision dataset.
GenDis: Generative-Discriminative Dual-View Co-Training for Generalized Category Discovery (2026.acl-long)

Copied to clipboard

Challenge: Existing methods rely on one-hot discriminative supervision, leading to overfitting on seen classes and poor generalization to unseen ones.
Approach: They propose a Generative–Discriminative Dual-View Co-Training framework that unifies discriminative classification and semantic label generation within an LLM.
Outcome: The proposed framework outperforms existing methods on five benchmarks on the generalized category discovery (GCD) task.
Adversarial Preference Learning for Robust LLM Alignment (2025.findings-acl)

Copied to clipboard

Challenge: Modern language models rely on Reinforcement Learning from Human Feedback (RLHF) to encourage safe behaviors, but they remain vulnerable to adversarial attacks due to three key limitations: (1) the inefficiency and high cost of human annotation; (2) the vast diversity of potential adversarials; and (3) the risk of feedback bias and reward hacking.
Approach: They propose an iterative adversarial training method that incorporates three key innovations to address these challenges.
Outcome: Experiments on Mistral-7B-Instruct-v0.3 show that the proposed method significantly enhances robustness and reduces harmful outputs from 5.88% to 0.43%.
Toward Optimal LLM Alignments Using Two-Player Games (2025.findings-emnlp)

Copied to clipboard

Challenge: Alignment of large language models (LLM) is a process that ensures the model’s responses to user prompts align with human intentions and social values.
Approach: They propose an alignment method based on a two-agent game consisting of an adversarial agent and a defensive agent.
Outcome: The proposed method improves on a two-agent game with an adversarial agent and a defensive agent.
AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to verify agent behaviors in complex environments rely on rule-based verifiers or LLM-as-a-Judge models.
Approach: They propose a benchmark to evaluate Agent-as-a-Judge across three domains . the benchmark covers search, data systems, and graphical user interfaces - with 155 tasks and 516 trajectories .
Outcome: The proposed benchmark outperforms existing benchmarks in search, data systems, and GUI domains while revealing open challenges in agent-based verification.
Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimization (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to training large language models suffer from unstable value estimation, whereas outcome supervision struggles with credit assignment due to sparse, trajectory-level rewards.
Approach: They propose a framework that integrates process supervision into group relative policy optimization.
Outcome: The proposed framework outperforms standard GRPO on knowledge-intensive benchmarks by 5.0% and 6.3% on Qwen3-1.7B.
EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models (2024.acl-demos)

Copied to clipboard

Challenge: Large Language Models (LLMs) suffer from knowledge cutoff or fallacy issues, which means they are unaware of unseen events or generate text with incorrect facts owing to outdated/noisy data.
Approach: They propose an easy-to-use knowledge editing framework for Large Language Models that allows users to easily edit updated knowledge and adjust undesired behavior while minimizing the impact on unrelated inputs.
Outcome: The proposed framework surpasses traditional fine-tuning in terms of reliability and generalization.
Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts (2025.emnlp-main)

Copied to clipboard

Challenge: Recent models have extended Corresponding Author. context lengths to millions of tokens while maintaining reasoning and comprehension capabilities.
Approach: They propose a benchmark to evaluate the ability of large language models to extract sequential information items from long contexts.
Outcome: The proposed model achieves maximum accuracy of 63.50% on six well-known LLMs.
Beyond Inherent Cognition Biases in LLM-Based Event Forecasting: A Multi-Cognition Agentic Framework (2025.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models exhibit human-like cognitive biases in event forecasting . a human-curated dataset reveals significant cognitive bias in LLMs .
Approach: They propose a human-curated dataset to explore LLMs' cognitive biases . they leverage LLM participants to act as multi-cognition event participants .
Outcome: The proposed framework alleviates cognitive biases in LLMs and offers diverse perspectives.
LLMs May Perform MCQA by Selecting the Least Incorrect Option (2025.coling-main)

Copied to clipboard

Challenge: Multiple Choice Question Answering (MCQA) is a fundamental format for various tasks in NLP, such as commonsense reasoning.
Approach: They propose a method to increase the number of correct options in a dataset.
Outcome: The proposed method improves the performance of multiple choice question answering (MCQA) and improves its accuracy.
GrandGuard: Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety (2026.findings-acl)

Copied to clipboard

Challenge: a survey of older adults shows that many LLMs mishandle elderly-specific contextual risks.
Approach: They propose a framework to assess elderly-specific contextual risks in LLM interactions . they use a taxonomy to identify 50 fine-grained risk types across mental well-being, financial, medical, toxicity, and privacy domains .
Outcome: a new framework assesses elderly-specific contextual risks in LLM interactions . it achieves 96.2% and 90.9% unsafe-prompt detection accuracy, respectively .
Targeted Distillation for Sentiment Analysis (2025.emnlp-main)

Copied to clipboard

Challenge: Recent studies demonstrate that large language models exhibit remarkable capabilities and achieve state-of-the-art performance in diverse sentiment analysis tasks.
Approach: They propose a distillation framework that decouples knowledge from alignment and introduces a sentiment analysis benchmark that covers a diverse set of tasks.
Outcome: The proposed framework improves models' generalization to unseen tasks and their generalization is strong against existing small-scale models.
The CRECIL Corpus: a New Dataset for Extraction of Relations between Characters in Chinese Multi-party Dialogues (2022.lrec-1)

Copied to clipboard

Challenge: Existing datasets focus on relation extraction between two entities in one sentence, and some focus on cross-sentence relationships.
Approach: They propose to use a Chinese multi-party dialogue dataset for automatic extraction of dialogue-based character relationships.
Outcome: The proposed dataset extracts relationships between 140 entities on the CRECIL corpus and another existing relation extraction corpus.
MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning (2026.findings-acl)

Copied to clipboard

Challenge: Existing work on long document visual question answering is based on Retrieval-Augmented Generation (RAG) where textual or visual content is encoded into embeddings and relevance is determined by similarity scores with respect to the original query.
Approach: They propose a framework that employs an agentic, vision-aware workflow to address long document visual question answering through iterative information discovery and synthesis.
Outcome: The proposed framework outperforms existing RL systems by 10.4% on the MMLongbench-Doc benchmark and demonstrates superior training performance over GRPO.
LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation (2025.findings-emnlp)

Copied to clipboard

Challenge: Current medical benchmarks have limitations in question design, data sources and evaluation methods.
Approach: They propose a new benchmark covering five core medical areas . it includes 2,996 questions created from real-world electronic health records .
Outcome: The proposed model covers five core medical areas and includes 2,996 questions created from real-world electronic health records and expert-designed clinical scenarios.
TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use (2025.findings-emnlp)

Copied to clipboard

Challenge: a new approach to training large language models (LLMs) overlooks task-specific characteristics in tool use, leading to performance bottlenecks.
Approach: They propose a task-feature-based framework that mitigates the effects of suboptimal training data . they use a dataset to train large-scale LLMs and a reward mechanism tailored to error categories .
Outcome: The proposed framework matches or surpasses open- and closed-source LLMs in tool-use performance using only 1,217 training data points.
Pragmatic Inference Chain (PIC) Improving LLMs’ Reasoning of Authentic Implicit Toxic Language (2025.emnlp-main)

Copied to clipboard

Challenge: Existing studies have shown that LLMs can detect toxicity by using a variety of inference-intensive tasks, such as understanding humour and metaphors.
Approach: They propose a new method to prompt LLMs to identify toxic language using a set of online data that are verified by human annotators.
Outcome: The proposed method significantly improves the success rate of GPT-4o, Llama-3.1-70B-Instruct, DeepSeek-v2.5, and DeepSeq-v3 in identifying implicit toxic language compared to five baseline prompts, such as CoT and rule-based baselines.
EmotionTalk: An Interactive Chinese Multimodal Emotion Dataset With Rich Annotations (2026.findings-acl)

Copied to clipboard

Challenge: Existing datasets face issues such as low quality, limited scale, and incomplete modalities, hindering model performance.
Approach: They propose to use Chinese multimodal datasets to capture authentic emotional interplay from 19 professional actors.
Outcome: The EmotionTalk dataset spans 23.6 hours of dyadic conversations across diverse scenarios.
GBV-SQL: Guided Generation and SQL2Text Back-Translation Validation for Multi-Agent Text2SQL (2026.acl-long)

Copied to clipboard

Challenge: Existing frameworks for Text2SQL generation still have a critical semantic gap . a dedicated validator translates generated SQL back into natural language and checks whether its logic is aligned with the original question.
Approach: They propose a framework that introduces Guided Generation with SQL2Text Back-translation Validation . dedicated validator translates generated SQL back into natural language and checks whether logic is aligned with original question .
Outcome: The proposed framework achieves 63.23% execution accuracy on the BIRD benchmark and 90.42% on repaired BIDR dev.
Parrot: A Training Pipeline Enhances Both Program CoT and Natural Language CoT for Reasoning (2025.emnlp-main)

Copied to clipboard

Challenge: Existing work focuses on enabling models to generate natural language chain-of-thought rationales or leverage executable and verifiable code, such as Python.
Approach: They propose a novel training pipeline that integrates sequential P-CoT and N-Co T generation and a subtask hybrid training strategy to facilitate natural language transferability.
Outcome: The proposed training pipeline improves both N-CoT and P-Co T performance over the RL baseline.
On Continual Model Refinement in Out-of-Distribution Data Streams (2022.acl-long)

Copied to clipboard

Challenge: Existing continual learning (CL) problems cannot cover real-world scenarios such as out-of-distribution errors.
Approach: They propose a continual model refinement problem formulation to solve this problem . they extend several existing continual learning approaches to the CMR problem based on a general sampling algorithm .
Outcome: The proposed model refinement solution improves on existing models and their performance metrics.
Diversity in Unity, Theory in Practice: Hierarchical Multitask Benchmarks for Chinese Minority Languages (2026.acl-long)

Copied to clipboard

Challenge: CMiLBench is a framework to evaluate linguistically and culturally diverse minority languages . rapid evolution of LLMs has revolutionized NLP, but progress is unevenly distributed .
Approach: They propose a framework to translate a theoretical notion of "diversity in unity" into practical evaluation for three minority languages . CMiLBench comprises 24,663 instances across 5 difficulty levels and 17 tasks .
Outcome: The proposed framework evaluates 14 state-of-the-art LLMs with a hybrid framework . it integrates automatic metrics and LLM-as-a-Judge scoring .
AEGIS: A Holistic Benchmark for Evaluating Forensic Analysis of AI-Generated Academic Images (2026.acl-long)

Copied to clipboard

Challenge: AEGIS examines whether current models can effectively audit AI-generated images in academic papers.
Approach: They propose a holistic benchmark for forensic analysis of AI-Generated academic ImageS that reveals limitations in academic image forensics.
Outcome: AEGIS compared with existing benchmarks on seven academic categories and features key advances in forensic analysis.
RoCoIns: Enhancing Robustness of Large Language Models through Code-Style Instructions (2024.lrec-main)

Copied to clipboard

Challenge: Large Language Models (LLMs) have shown remarkable capabilities in following human instructions and solving NLU tasks.
Approach: They propose to use code style instructions to replace typically natural language instructions to provide more precise instructions and strengthen the robustness of LLMs.
Outcome: The proposed method outperforms natural language models on eight robustness datasets and achieves an improvement of 5.68% in test set accuracy and a reduction of 5.66 points in Attack Success Rate (ASR).
A Probabilistic Toolkit for Multi-grained Word Segmentation in Chinese (2025.coling-demos)

Copied to clipboard

Challenge: Existing tools for word segmentation are based on different linguistic theories or target different scenarios.
Approach: They propose a probabilistic toolkit for multi-grained word segmentation in Chinese . they adopt semi-Markov CRF for single-grain word segmenting (SWS) .
Outcome: The proposed approach can produce marginal probabilities of words during inference and significantly improve performance in the cross-domain scenario.
Benchmarking the Fine-Grained Discriminability in Image-Text Retrieval via Controlled Contrastive Differences (2026.findings-acl)

Copied to clipboard

Challenge: Existing cross-modal image-text retrieval models often retrieve samples with inconsistent details.
Approach: They propose two fine-grained image-text retrieval benchmarks that incorporate extensive contrastive samples with one controlled contrastive difference from its anchor.
Outcome: Extensive experiments show that contrastive samples can significantly degrade retrieval performance.
ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development (2026.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) have redefined the role of AI in software engineering . current benchmarks focus on localized code generation, but neglect dynamic, full-process requirements of real-world engineering.
Approach: They propose a benchmark to evaluate agentic backend coding within a realistic, executable workflow.
Outcome: The ABC-Bench benchmark evaluates agentic backend coding within a realistic, executable workflow.
Analyzing values about gendered language reform in LLMs’ revisions (2025.emnlp-main)

Copied to clipboard

Challenge: In the past years, LLMs have been used in everyday tasks, especially the formulation and revision of text.
Approach: They examine LLMs' revision of gendered role nouns and their justifications using a prompt set-up to examine their alignment with feminist and trans-inclusive language reforms for English.
Outcome: The proposed revision choices are based on the literature and empirical evidence.
ComRAG: Retrieval-Augmented Generation with Dynamic Vector Stores for Real-time Community Question Answering in Industry (2025.acl-industry)

Copied to clipboard

Challenge: Existing methods for Community Question Answering (CQA) focus on static knowledge, limiting their applicability to real-world scenarios.
Approach: They propose a retrieval-augmented generation framework for real-time industrial CQA that integrates static knowledge with dynamic historical QA pairs via a centroid-based memory mechanism.
Outcome: The proposed framework outperforms baselines on three industrial CQA datasets and achieves 25.9% improvement in vector similarity, reducing latency by 8.7%–23.3%, and lowering chunk growth from 20.23% to 2.06% over iterations.
StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback (2024.acl-long)

Copied to clipboard

Challenge: Existing work integrates reinforcement learning with compiler feedback to enhance code generation quality but the long code generated by LLMs makes RL exploration ineffective.
Approach: They propose a framework that integrates reinforcement learning and compiler feedback to enhance code generation quality.
Outcome: The proposed framework outperforms state-of-the-art approaches in corresponding benchmarks and integrates reinforcement learning with compiler feedback to improve code generation quality.
Sketch-Driven Regular Expression Generation from Natural Language and Examples (2020.tacl-1)

Copied to clipboard

Challenge: Recent systems for converting natural language descriptions into regexes have achieved some success, but typically deal with short, formulaic text and can only produce simple regexe.
Approach: They propose a framework for regex synthesis in a context where both natural language and examples are available.
Outcome: The proposed framework achieves state-of-the-art on two prior datasets and a real-world dataset, which existing neural systems completely fail on.
Rethinking Stateful Tool Use in Multi-Turn Dialogues: Benchmarks and Challenges (2025.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks that assess Language Models (LMs) as Language Agents (LAs) for tool use focus on stateless, single-turn interactions or partial evaluations, overlooking the inherent stateful nature of interactions in multi-turn applications.
Approach: They propose a multi-turn dialogue dataset with stateful tool interactions considering the whole life cycle of tool use across six key tasks in three stages . they also build VirtualMobile – an embodied virtual mobile evaluation environment to simulate API calls and assess the robustness of the created APIs.
Outcome: The proposed dataset evaluates 13 open- and closed-source LLMs and provides detailed analysis at each stage.
Stand on The Shoulders of Giants: Building JailExpert from Previous Attack Experience (2025.emnlp-main)

Copied to clipboard

Challenge: Existing methods to generate human-aligned content with a “jailbreak prompt” are inefficient and repetitive, causing inefficiency and a lack of experience.
Approach: They propose a framework that integrates past attack experiences to aid current jailbreak attempts.
Outcome: The proposed framework improves both attack effectiveness and efficiency compared to the current black-box jailbreak method.
SlideCoder: Layout-aware RAG-enhanced Hierarchical Slide Generation from Design (2025.emnlp-main)

Copied to clipboard

Challenge: Existing natural language-based LLM generation methods struggle to capture visual and structural nuances of slide designs.
Approach: They propose a layout-aware framework for generating editable slides from reference images . they propose python code that translates NL instructions into Python code to construct each slide .
Outcome: The proposed framework outperforms state-of-the-art models by up to 40.5 points . it also outperformed open-source models with improved reverse-engineered data.
VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Training (2026.acl-long)

Copied to clipboard

Challenge: Reinforcement Learning (RL) in real-world environments often suffers from ambiguous or incomplete supervision.
Approach: They propose a framework that enhances value modeling for robust RL in LLM post-training by integrating auxiliary losses guided by entropy and perplexity from a frozen language model and variational information bottleneck.
Outcome: The proposed framework outperforms baselines on multi-turn dialogue, math reasoning, and science QA with rule-based and model-based rewards.
MatchPrompt: Prompt-based Open Relation Extraction with Semantic Consistency Guided Clustering (2022.emnlp-main)

Copied to clipboard

Challenge: Existing methods for open relation extraction (OpenRE) focus on labeled and pre-defined instances, which are costly to acquire in reality.
Approach: They propose a framework that can extract relations without pre-defined types from open-domain corpus with efficient knowledge transfer from a few pre-determined relational instances.
Outcome: The proposed framework achieves the new SOTA results for OpenRE on different datasets.
Dialect-SQL: An Adaptive Framework for Bridging the Dialect Gap in Text-to-SQL (2025.emnlp-main)

Copied to clipboard

Challenge: Existing Text-to-SQL research focuses on specific database systems, limiting adaptability to different dialects.
Approach: They propose a framework that employs Object Relational Mapping (ORM) code as an intermediate language to bridge this gap.
Outcome: The proposed framework outperforms existing methods that generate SQL queries directly.
Self-supervised Preference Optimization: Enhance Your Language Model with Preference Degree Awareness (2024.findings-emnlp)

Copied to clipboard

Challenge: Recent studies have focused on replacing the reward model in Reinforcement Learning with Human Feedback (RLHF) methods for Large Language Models (LLMs).
Approach: They propose a self-supervised preference optimization framework that replaces the reward model with a preference loss and alignment loss to improve LLMs' ability to understand human preferences.
Outcome: The proposed framework can be integrated with existing preference optimization methods and significantly boost their performance.
From Scores to Preferences: Redefining Evaluation Paradigm for Speech Quality Reward Modeling (2026.findings-acl)

Copied to clipboard

Challenge: Experimental results show that the MOS-aware GRM significantly improves fine-grained speech quality discrimination.
Approach: They propose a MOS-aware reward model that incorporates MOS gap into reward function during reinforcement learning.
Outcome: The proposed model significantly improves fine-grained speech quality discrimination.
Detecting Customer Complaint Escalation with Recurrent Neural Networks and Manually-Engineered Features (N19-2)

Copied to clipboard

Challenge: e-commerce companies often have the option of escalating complaints by filing grievances with a government authority . this is detrimental to an ecommerce company, but this problem is challenging to solve by integrating recurrent neural networks with manually-engineered features.
Approach: They propose a model that integrates recurrent neural networks with manually-engineered features to identify cases where the customer expresses such an intent.
Outcome: The proposed model outperforms baseline models and provides better recall and triage for specialized agents.
Easy and Efficient Transformer: Scalable Inference Solution For Large NLP Model (2022.naacl-industry)

Copied to clipboard

Challenge: Recent studies show that transformer-based models are effective over many tasks, but they are expensive to deploy in the industrial application.
Approach: They propose a transformer-based inference solution that optimizes kernels for long inputs and large hidden sizes and a flexible CUDA memory manager to reduce the memory footprint when deploying a large model.
Outcome: The proposed solution achieves an average speedup of 1.40-4.20x on the transformer decoder layer with an A100 GPU.
HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval-Augmented Generation (2025.findings-acl)

Copied to clipboard

Challenge: Traditional retrieval systems focus on lexical or semantic similarity rather than logical relevance.
Approach: They propose a new RAG framework that augments retrieval with logical reasoning . hopRAG uses a retrieve-reason-prune mechanism to explore multi-hop neighbors .
Outcome: The proposed framework outperforms conventional retrieval systems and state-of-the-art benchmarks on multi-hop QA tasks.
ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use (2025.acl-long)

Copied to clipboard

Challenge: Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models.
Approach: They propose a dataset that provides rigorous evaluation of multi-hop tool use.
Outcome: The proposed model achieves 49.04% accuracy across five model families.
SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge Refinement (2025.acl-short)

Copied to clipboard

Challenge: Using Large Language Models (LLMs)-based agents can enhance their understanding of environments and tasks.
Approach: They propose a framework that allows agents to synthesize possible scenarios with multi-step action invocation within the action space and perform Monte Carlo Tree Search exploration to refine their action knowledge in the current environment.
Outcome: The proposed framework synthesizes possible scenarios with multi-step action invocation within the action space and performs Monte Carlo Tree Search exploration to refine action knowledge in the current environment.
Characteristic AI Agents via Large Language Models (2024.lrec-main)

Copied to clipboard

Challenge: Commercial products have been devoted to creating character-driven chatbots using large language models, but academic research in this area remains relatively scarce.
Approach: They investigate the performance of LLMs in constructing characteristic AI agents by simulating real-life individuals across different settings.
Outcome: The proposed benchmark compared LLMs with real-life individuals in different settings and includes evaluation metrics.
SampleMix: A Sample-wise Pre-training Data Mixing Strategy by Coordinating Data Quality and Diversity (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for pretraining data mixing for large language models neglect significant inter-domain overlaps and commonalities, failing to control the global diversity of the constructed training dataset.
Approach: They propose a sample-wise data mixture approach that performs global cross-domain sampling by systematically evaluating the quality and diversity of each sample.
Outcome: The proposed method exceeds existing domain-based methods in multiple downstream tasks and perplexity assessments.
Transparent and Scrutable Recommendations Using Natural Language User Profiles (2024.acl-long)

Copied to clipboard

Challenge: Recent advances in Large Language Models (LLMs) rely on implicit or explicit feedback from users to suggest new items, resulting in a lack of transparency and a user's ability to scrutinize and modify their preferences.
Approach: They propose to use a natural language (NL) user profile to summarize a user's preferences and then use it to fine-tune a LLM using only NL profiles to make transparent and scrutable recommendations.
Outcome: The proposed model performs on two benchmarking rating prediction datasets and is comparable to existing models.
JPU: Bridging Jailbreak Defense and Unlearning via On-Policy Path Rectification (2026.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) are vulnerable to diverse jailbreak attacks despite extensive safety alignment .
Approach: They propose a method to rectify dynamic jailbreak paths towards safety anchors by dynamically mining on-policy adversarial samples to expose vulnerabilities and identify jailbreak path.
Outcome: The proposed model significantly improves jailbreak resistance against dynamic attacks while maintaining its utility.
TrendFact: A Benchmark Towards Hotspot Perception in Automatic Fact-Checking (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks lack social metadata and evaluation framework to meet this urgent evaluation needs.
Approach: They propose a benchmark capable of evaluating HPA and three fact-checking tasks.
Outcome: The proposed framework improves HPA and computational efficiency for RLM-driven systems.
ASCM: An Answer Space Clustered Prompting Method without Answer Engineering (2022.findings-acl)

Copied to clipboard

Challenge: Pre-trained language models have shown a great impact on NLP tasks.
Approach: They propose an answer space clustered prompting model and a synonym initialization method that automatically categorizes all answer tokens in a semantic-clustered embedding space.
Outcome: Experiments show that the proposed method outperforms existing state-of-the-art methods in few-shot settings.
Prompt Combines Paraphrase: Teaching Pre-trained Models to Understand Rare Biomedical Words (2022.coling-1)

Copied to clipboard

Challenge: Pre-trained models perform poorly with limited data and rare biomedical words.
Approach: They propose to use prompt to fine-tune pre-trained models for biomedical domain tuning with a simple approach.
Outcome: The proposed method achieves up to 6% improvement in biomedical natural language inference task without any extra parameters or training steps using few-shot vanilla prompt settings.
HoLLMwood: Unleashing the Creativity of Large Language Models in Screenwriting via Role Playing (2024.findings-emnlp)

Copied to clipboard

Challenge: Generative AI has demonstrated unprecedented creativity in the field of computer vision, yet such phenomena have not been observed in the realm of literary creation.
Approach: They propose a framework for unleashing the creativity of large language models (LLMs) they assign LLMs to different roles involved in real-world scenario, they write .
Outcome: The proposed framework outperforms baselines in terms of coherence, relevance, interestingness and overall quality on automatically generated screenplays.
Context-Driven Index Trimming: A Data Quality Perspective to Enhancing Precision of RALMs (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing research often overlooks the data quality issues within retrieval results, often caused by inaccurate existing vector-distance-based retrieval methods.
Approach: They propose to use Context-Driven Index Trimming framework to capture and regulate consistency between retrieved contexts and modify indexes in the database.
Outcome: Experiments show that the proposed framework can improve answer quality by 3.75% on open-domain question-answering tasks.
A Neural Network Based Model for Loanword Identification in Uyghur (L18-1)

Copied to clipboard

Challenge: Lexical borrowing happens in almost all languages, and we propose a new method to identify loanwords in Uyghur.
Approach: They propose a neural network based loanword identification model for Uyghur that captures past and future information and learns both word level and character level features automatically.
Outcome: The proposed model outperforms baseline models on Chinese, Arabic and Russian loanword detection in Uyghur.
QuadrupletBERT: An Efficient Model For Embedding-Based Large-Scale Retrieval (2021.naacl-main)

Copied to clipboard

Challenge: Existing methods for large-scale query-document retrieval are expensive and require sparse handcrafted features.
Approach: They propose a quadrupletBERT model for effective and efficient retrieval using pre-trained language models like BERT.
Outcome: The proposed model improves retrieval phase and leverages distances between simple negative and hard negative instances to obtain better embeddings.
Knowledge Injection Exists in MoE? Exploring Expert-Aware Contrast Decoding in MoE for Mitigating LLMs’ Hallucinations (2026.acl-long)

Copied to clipboard

Challenge: Existing methods to mitigate hallucinations include prompt engineering and model optimization, but lack domain generalization and potential errors in fine-tuning data may exacerbate the hallucism.
Approach: They propose an expert-aware adaptive contrast decoding that uses expert differences in MoE’s higher layers to mitigate hallucinations on QA tasks.
Outcome: The proposed method outperforms baseline models on four datasets Large language models (LLMs) show strong performance but suffer from hallucinations, limiting their application.
Adaptive Retrieval-Augmented Generation for Conversational Systems (2025.findings-naacl)

Copied to clipboard

Challenge: Existing studies have shown the effectiveness of retrieving and augmenting external knowledge for informative responses.
Approach: They propose to use a gating model to predict if a conversational system requires retrieval-augmented generation to generate high-quality responses with high confidence.
Outcome: The proposed model can predict if a conversational system requires RAG to generate high-quality responses with high confidence.
Mitigating Biases of Large Language Models in Stance Detection with Counterfactual Augmented Calibration (2025.naacl-long)

Copied to clipboard

Challenge: Large language models generate biased stances due to spurious correlations and preference towards certain individuals and topics.
Approach: They propose a counterfactual Augmented Calibration Network to calibrate potential bias in stance detection of large language models.
Outcome: The proposed calibration network can mitigate biases of large language models, achieving state-of-the-art results.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations