Papers by Jiaxin Zhang

73 papers
The Elephant in the Room: Exploring the Role of Neutral Words in Language Model Group-Agnostic Debiasing (2025.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) are increasingly integrated into our daily lives, raising ethical concerns, especially about perpetuating stereotypes.
Approach: They propose a method that incorporates a neutral word semantics-based loss function to alleviate the deterioration of the LMS during debiasing.
Outcome: The proposed method alleviates the deterioration of the Language Modeling Score (LMS) by incorporating a neutral word semantics-based loss function.
Failures are Treasures: Constructing a Pedagogical Bridge for Agentic Strategy Distillation (2026.findings-acl)

Copied to clipboard

Challenge: Existing knowledge distillation methods focus on imitating successful trajectories, whereas small language models are fragile and often collapsing after encountering errors.
Approach: They propose a Pedagogical Bridge for Reflective Insight and Distillation of Guiding Errors that combines reflection-in-action and reflection-on-action to enable agents to diagnose and correct critical errors while abstracting transferable strategies from contrastive student–teacher trajectories.
Outcome: Experiments show that the proposed model significantly elevates performance in large language models (SLMs) .
ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations (2020.findings-emnlp)

Copied to clipboard

Challenge: Experimental results show that pre-trained text encoders can perform many NLP tasks with less resource.
Approach: They propose a BERT-based Chinese text encoder enhanced by n-gram representations . they show reasonable performance when ZEN is trained on a small corpus .
Outcome: The proposed encoder incorporates the comprehensive information of both the character sequence and words or phrases it contains.
Confidence-Aware Reasoning: Optimizing Self-Guided Thinking Trajectories in Large Reasoning Models (2025.emnlp-industry)

Copied to clipboard

Challenge: Chain-of-thought enables large reasoning models to reason through multi-step problems but often leads to unnecessarily long or redundant reasoning traces, a phenomenon known as overthinking.
Approach: They propose an inference-time framework that selectively prunes low-utility reasoning blocks and halts early when sufficient confidence has been achieved.
Outcome: The proposed framework improves answer accuracy by up to +13.3% while reducing average reasoning length by 40%–50%.
G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks (2022.emnlp-main)

Copied to clipboard

Challenge: Existing domain-adaptive pre-training (DAPT) models tend to forget the general knowledge acquired by general PLMs, leading to catastrophic forgetting and sub-optimal performance.
Approach: They propose a framework which augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge.
Outcome: The proposed framework augments the domain-specific PLM by a memory built from the frozen general PLM without losing the general knowledge.
From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Large Language Models (LLMs) have remarkable capabilities, but unreliability remains a barrier to deployment in high-stakes domains.
Approach: They propose to transform uncertainty from a passive diagnostic metric to an active control signal guiding real-time model behavior.
Outcome: The proposed model evolution from passive diagnostic metric to active control signal is critical for high-stakes applications.
Learning from Mistakes: Iterative Prompt Relabeling for Text-to-Image Diffusion Model Training (2024.findings-emnlp)

Copied to clipboard

Challenge: Recent advances in diffusion models have shown impressive performance in many domains, but their ability to follow instructions is still unsatisfactory.
Approach: They propose an algorithm that aligns images to text through iterative image sampling and prompt relabeling with feedback.
Outcome: The proposed algorithm improves on the spatial relation VISOR benchmark by 15.22% compared to previous methods.
Multiplex Word Embeddings for Selectional Preference Acquisition (D19-1)

Copied to clipboard

Challenge: Existing word embeddings are limited in their ability to represent fixed vectors . instead, they incorporate relational dependencies of different words into their embeddables - a limitation that is addressed by a multiplex model .
Approach: They propose a word embedding model which incorporates relational dependencies of different words into their embeddables.
Outcome: The proposed model can be easily extended according to various relations among words.
On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directions (2025.findings-emnlp)

Copied to clipboard

Challenge: Conceptualization is a fundamental element of human cognition and plays a pivotal role in generalizable reasoning.
Approach: They propose to categorize different types of conceptualizations into four levels based on the types of instances being conceptualized.
Outcome: The proposed categorization of different types of conceptualizations into four levels based on the types of instances being conceptualized .
LLMs Trust Humans More, That’s a Problem! Unveiling and Mitigating the Authority Bias in Retrieval-Augmented Generation (2025.acl-long)

Copied to clipboard

Challenge: Large language models (LLMs) generate outputs that stray from user input or contravene established knowledge.
Approach: They propose a new phenomenon, Authority Bias, where LLMs favor one knowledge source over the other . they propose atomic information that generates conflicts and a Conflict Detection Enhanced Query framework .
Outcome: The proposed framework reduces Authority bias in large language models . it detects conflicts, performs credibility assessment on conflicting paragraphs, and detects perturbed text .
Hybrid and Collaborative Passage Reranking (2023.findings-acl)

Copied to clipboard

Challenge: Existing solutions to passage reranking focus on enriching the interaction between query and each passage separately, neglecting the context among the top-ranked passages.
Approach: They propose a Hybrid and Collaborative Passage Reranking method that leverages the similarity measurements of upstream retrievers for passage collaboration.
Outcome: Experiments show that HybRank improves over existing methods and improves performance.
Don’t Stop Early: Scalable Enterprise Deep Research with Controlled Information Flow and Evidence-Aware Termination (2026.acl-industry)

Copied to clipboard

Challenge: Enterprise deep research systems fail to produce decision-ready reports due to uneven information coverage, context explosion, and premature stopping.
Approach: They propose a scalable Enterprise Deep Research (EDR) architecture that decomposes requests into coverage-driven objectives via outline generation with reflection and localizes context with dependency-guided execution and explicit information sharing.
Outcome: The proposed system achieves the strongest overall performance compared with competitive deep-research baselines on internal sales enablement tasks and the public DeepResearch Bench benchmark.
Why Does New Knowledge Create Messy Ripple Effects in LLMs? (2024.emnlp-main)

Copied to clipboard

Challenge: Existing research has focused on post-training knowledge editing (KE) for language models to ensure that knowledge remains accurate and up-to-date.
Approach: They propose to use a GradSim indicator to detect when and why updated knowledge ripples in language models.
Outcome: The proposed indicator GradSim shows that LMs that fail to handle ripple effects have low GradSIM.
Text Classification Using Label Names Only: A Language Model Self-Training Approach (2020.emnlp-main)

Copied to clipboard

Challenge: Current text classification methods require a large number of labeled documents as training data.
Approach: They propose a model that uses only the label name of each class to train classification models on unlabeled data without using any labeled examples.
Outcome: The proposed model achieves 90% accuracy on four benchmark datasets using label names as the only supervision .
Gradient-guided Attention Map Editing: Towards Efficient Contextual Hallucination Mitigation (2025.findings-naacl)

Copied to clipboard

Challenge: Large Language Models (LLMs) often experience “contextual hallucination” where they prioritize self-generated content over input context, leading to a disregard for pertinent details.
Approach: They propose a method that dynamically adjusts attention maps to enhance contextual relevance by using a trained classifier to identify attention maps likely to induce hallucinations.
Outcome: The proposed approach reduces hallucinations across open-source models on summarization and open-book QA tasks.
SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for hallucination detection rely on self-consistency check alone . prominent LMs exhibit a tendency to produce exceedingly confident, but erroneous, assertions .
Approach: They propose a sampling-based method that expands on the principle of self-consistency checking to detect hallucinations at question-level and model-level.
Outcome: The proposed method outperforms the state of the art in detecting non-factual and factual statements across multiple question-answering and open-domain generation benchmarks.
Joint Coreference Resolution and Character Linking for Multiparty Conversation (2021.eacl-main)

Copied to clipboard

Challenge: Character linking is the task of linking mentioned people in conversations to the real world . human use of pronouns or normal entities makes it difficult to link mentioned people to real people . a critical step towards understanding conversations is grounding mentioned people - a goal of the natural language processing community .
Approach: They propose to integrate richer context from the coreference relations among different mentions to help the linking task.
Outcome: The proposed model outperforms all previous models on both tasks.
LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces (2026.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks for agentic programming in long-horizon command-line interface tasks are limited by short task horizons, data contamination from GitHub scraping, and a lack of fine-grained evaluation metrics.
Approach: They propose a benchmark to evaluate agentic capabilities across long-horizon command-line interface tasks.
Outcome: The proposed benchmarks cover four engineering categories: from scratch, feature addition, bug fixing, and refactoring.
emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation (2024.findings-acl)

Copied to clipboard

Challenge: Existing models for speech emotion recognition are not suitable for emotional tasks.
Approach: They propose a universal speech emotion representation model that is pre-trained on open-source emotion data.
Outcome: euphoria2vec outperforms state-of-the-art models and emotion specialist models . it shows consistent improvements among 10 different languages of speech emotion recognition datasets .
Probabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questions (2023.findings-emnlp)

Copied to clipboard

Challenge: Large language models (LLMs) are capable of answering knowledge-intensive complex questions with chain-of-thought reasoning.
Approach: They propose a method to solve complex questions with a tree-of-thought approach using parametric knowledge and retrieved external knowledge to augment CoT reasoning.
Outcome: The proposed approach outperforms SOTA methods on three Complex QA datasets under the open-domain setting.
Synthetic Knowledge Ingestion: Towards Knowledge Refinement and Injection for Enhancing Large Language Models (2024.emnlp-main)

Copied to clipboard

Challenge: Large language models capture factual knowledge across a wide range of domains, but refining their capabilities on previously seen knowledge remains a challenge.
Approach: They propose a synthetic knowledge ingestion method that leverages fine-grained synthesis and interleaved generation to construct high-quality data representations from raw knowledge sources.
Outcome: The proposed method outperforms baseline methods on question-answering tasks spanning finance, biomedicine, and open-generation domains.
M³GQA: A Multi-Entity Multi-Hop Multi-Setting Graph Question Answering Benchmark (2025.acl-long)

Copied to clipboard

Challenge: GraphRAG systems have achieved remarkable progress in enhancing performance and reliability of large language models.
Approach: They propose a GraphRAG benchmark focusing on multi-entity queries with six settings for comprehensive evaluation.
Outcome: The proposed method can construct diverse data with semantically correct ground-truth reasoning paths.
RepEval: Effective Text Evaluation with LLM Representation (2024.emnlp-main)

Copied to clipboard

Challenge: Traditional metrics for automatic text evaluation are tailored to specific tasks, while LLM-based evaluation metrics are costly.
Approach: They propose a metric that leverages projections of LLM representations for evaluation.
Outcome: The proposed metric exhibits higher correlation with human judgments than previous methods on 14 datasets.
Can LLMs Hear the Dogwhistle? (2026.findings-acl)

Copied to clipboard

Challenge: Existing safety benchmarks focus on explicitly harmful content, but ignore context-dependent expressions such as dogwhistles.
Approach: They propose a benchmark for evaluating LLM safety under dogwhistle-driven prompts . their findings expose a blind spot in current safety evaluation practices .
Outcome: The proposed benchmark compared safety performance with toxic terms using dogwhistle-driven prompts.
Capture Human Disagreement Distributions by Calibrated Networks for Natural Language Inference (2022.findings-acl)

Copied to clipboard

Challenge: Previously, it's common to disregard it as noise or as a sign of poor-quality data, as their annotations are heavily based on personal experience and opinions.
Approach: They propose to capture the human disagreement distribution from the perspective of model calibration.
Outcome: The proposed model can achieve competitive performance when well-calibrated, on divergence scores between predictive probability and the true human opinion distribution, and the accuracy.
Chameleons and Guardians: Unveiling the Divergence in Personality Plasticity and Cognitive Resistance across LLMs (2026.findings-acl)

Copied to clipboard

Challenge: Existing studies on LLMs argue for its immutability, but prior studies have not found that personality-inducing contexts can be intentionally reshaped.
Approach: They propose a personality-inducing framework that reshapes LLMs via multi-agent collaboration . they paraphrase MBTI questions to create semantically equivalent but expressively diverse inducing contexts .
Outcome: Experiments on worldwide mainstream LLMs show that PIF transforms their original personalities into desired target personalities.
Improving Low-Resource Sequence Labeling with Knowledge Fusion and Contextual Label Explanations (2025.emnlp-main)

Copied to clipboard

Challenge: Existing approaches to sequence labeling are limited due to the scarcity of domain-specific data and semantic distribution biases in domain-based contexts.
Approach: They propose a framework that integrates an LLM-based knowledge enhancement workflow with a span-based Knowledge Fusion for Rich and Efficient Extraction model.
Outcome: The proposed model achieves state-of-the-art performance on multiple domain-specific sequence labeling datasets and is highly efficient.
MeepleLM: A Virtual Playtester Simulating Diverse Subjective Experiences (2026.acl-long)

Copied to clipboard

Challenge: Recent advances in large language models have expanded the role of board games as creative co-designers . however, current systems lack the capacity to offer constructive critique grounded in the emergent user experience .
Approach: They propose a large language model that internalizes persona-specific reasoning patterns to accurately simulate the subjective feedback of diverse player archetypes.
Outcome: The proposed model outperforms commercial models in community alignment and critique quality.
GR1: Reinforcement-Enhanced LLM for Geoscience Reasoning (2026.findings-acl)

Copied to clipboard

Challenge: Recent advances in large language models have demonstrated RL's substantial capacity to enhance multi-step reasoning beyond what supervised instruction tuning achieves.
Approach: They propose a framework that converts multimodal questions into descriptive text . they propose RL-enhanced geoscience reasoning that can be fine-tuned to a text-only level .
Outcome: The proposed framework improves accuracy and accuracy on multimodal questions while preserving answerability and difficulty.
CMMaTH: A Chinese Multi-modal Math Skill Evaluation Benchmark for Foundation Models (2025.coling-main)

Copied to clipboard

Challenge: Large language models excel in various language tasks, while large multimodal models effectively handle visual-language problems.
Approach: They propose to use a multimodal multimodal model evaluation benchmark to evaluate model performance in Chinese K12 classrooms.
Outcome: The proposed model evaluation tool is integrated with the CMMaTH dataset.
Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning (2024.findings-naacl)

Copied to clipboard

Challenge: Existing methods for surfacing symbolic reasoning capabilities are limited to narrow tasks . arithmetic computations are unnatural to perform in pure language space, and hence present difficulties for LLMs.
Approach: They propose a natural language embedded program framework for solving symbolic reasoning tasks.
Outcome: The proposed framework improves on strong baselines across math and symbolic reasoning, text classification, question answering, and instruction following tasks.
HyQE: Ranking Contexts with Hypothetical Query Embeddings (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches to rank contexts rely on similarity between contexts and queries, but these methods are limited by the number of candidate contexts.
Approach: They propose a scalable ranking framework that combines embedding similarity and large language models without fine-tuning.
Outcome: The proposed framework improves the performance across multiple benchmarks.
InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles (2025.emnlp-main)

Copied to clipboard

Challenge: Recent large language models (LLMs) have demonstrated strong reasoning abilities across complex mathematical and scientific domains.
Approach: They propose a framework to assess whether LLMs can capture and apply personalized reasoning styles in social deduction games.
Outcome: The proposed framework evaluates LLMs on the game Avalon and shows that they can capture and apply individualized reasoning styles.
Re-TASK: Revisiting LLM Tasks from Capability, Skill, and Knowledge Perspectives (2025.findings-acl)

Copied to clipboard

Challenge: Existing approaches to solving complex tasks with large language models (LLMs) fail to decompose tasks accurately or execute subtasks effectively.
Approach: They propose a Chain-of-Learning (CoL) paradigm that highlights task dependencies on specific capability items, further broken down into their constituent knowledge and skill components.
Outcome: The proposed model improves Yi-1.5-9B and Llama3-Chinese-8B for legal tasks by 45.00% and 24.50% on different domains.
CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (2025.findings-acl)

Copied to clipboard

Challenge: Image captioning has been a challenge for vision-language researchers for decades . current VLMs focus on tasks like visual question answering (YA) but image captioning is not as advanced as expected.
Approach: They evaluate VLMs' performance on image captioning using human annotations . they find that some metrics show high caption-level agreement with humans .
Outcome: The proposed model outperforms open-source models on image captioning . it achieves 93.4% correlation with human rankings at $4 per test .
AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora (2026.acl-long)

Copied to clipboard

Challenge: Existing knowledge graph construction frameworks require predefined schemas, limiting their scalability and domain coverage.
Approach: They propose a framework for fully autonomous knowledge graph construction that eliminates the need for predefined schemas.
Outcome: The proposed framework outperforms state-of-the-art models on multi-hop QA tasks and enhances LLM factuality.
Beyond Self-Report: Bridging the Intention-Behavior Gap in Critical Thinking Assessment via Interpretable Multi-Agent System (2026.acl-long)

Copied to clipboard

Challenge: Accurate assessment of critical thinking is limited by the Intention Behavior Gap in psychology . evaluators that measure self-reported competence are limited by multiagent architectures .
Approach: They propose a framework that operationalizes cognitive assessment into an interpretable multi-agent workflow with Assessment Chain-of-Thought.
Outcome: The proposed framework aligns better with human expert ratings than gold-standard inventories on large-scale simulations and human participants.
Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering (2023.acl-long)

Copied to clipboard

Challenge: Existing XQA methods focus on reasoning on a single knowledge source, e.g., structured knowledge bases, unstructured corpora, etc. Existing work in XQA focuses on integrating information from heterogeneous knowledge sources.
Approach: They propose to leverage question decomposing for heterogeneous knowledge integration by breaking down a complex question into simpler ones and selecting the appropriate knowledge source for each sub-question.
Outcome: The proposed framework outperforms SOTA methods on complex QA datasets.
Query2Particles: Knowledge Graph Reasoning with Particle Embeddings (2022.findings-naacl)

Copied to clipboard

Challenge: Existing methods to answer complex logical queries on incomplete knowledge graphs with missing edges are needed to solve the problem.
Approach: They propose a query embedding method that encodes queries and entities to the same embeddable space and then selects the answer entities based on similarities .
Outcome: The proposed method can answer complex logical queries on incomplete knowledge graphs with missing edges.
How Far are We from Robust Long Abstractive Summarization? (2022.emnlp-main)

Copied to clipboard

Challenge: Abstractive summarization has made tremendous progress in recent years . however, even under a short document setting, abstractive models often generate summaries that are repetitive, ungrammatical, and factually inconsistent with the source.
Approach: They perform fine-grained human annotations to evaluate long document abstractive summarization systems and develop factual consistency metrics.
Outcome: The proposed model can generate more relevant summaries but not factual ones.
How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answering (2024.findings-acl)

Copied to clipboard

Challenge: Recent studies have validated that large language models (LLMs) are capable of solving some KBQA problems, but there has been little discussion on the differences in LLMs’ proficiency in formal languages used in semantic parsing.
Approach: They propose to evaluate the understanding and generation ability of large language models (LLMs) to deal with differently structured logical forms by examining the inter-conversion of natural and formal language through in-context learning of LLMs.
Outcome: The proposed model can understand formal languages as well as humans, but generating correct logical forms remains a challenge.
Heuristic-based Search Algorithm in Automatic Instruction-focused Prompt Optimization: A Survey (2025.findings-acl)

Copied to clipboard

Challenge: Recent advances in Large Language Models (LLMs) have led to remarkable achievements across a variety of NLP tasks.
Approach: They propose a taxonomy of automatic prompt optimization methods that explore and improve prompts with minimal human oversight.
Outcome: The proposed methods can explore and improve prompts with minimal human oversight.
When Phrases Meet Probabilities: Enabling Open Relation Extraction with Cooperating Large Language Models (2024.acl-long)

Copied to clipboard

Challenge: Existing clustering-based open relation extraction methods use pre-trained language models . embeddings from language models are high-dimensional and anisotropic, so there is a gap .
Approach: They propose a framework that makes two LLMs work collaboratively to achieve clustering.
Outcome: The proposed framework outperforms existing methods by 1.4%3.13% on different datasets.
ETHICIST: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimation (2023.acl-long)

Copied to clipboard

Challenge: Recent studies show that pre-trained language models memorize a considerable fraction of training data, leading to privacy risk of information leakage.
Approach: They propose a method for targeted training data extraction using a smoothed soft prompting and calibrated confidence estimation.
Outcome: The proposed method significantly improves the extraction performance on a recently proposed public benchmark.
MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process Errors Identification (2025.findings-acl)

Copied to clipboard

Challenge: Existing benchmarks of large language models focus on error detection, neglecting other scenarios like reasoning search.
Approach: et al. propose a multi-task, multimodal benchmark to assess effectiveness of PRMs . step correctness, answers aggregation and reasoning process search are evaluated . ethical principles of MPBench are based on a set of evaluation paradigms based in a text-based benchmark .
Outcome: a new benchmark assesses the effectiveness of large language models (LLMs) in multiple scenarios . it uses three evaluation paradigms to assess the effectiveness and compares them with existing models . a the proposed model improves reasoning accuracy by providing stepwise feedback for multi-step reasoning results .
Holistic Evaluation for Interleaved Text-and-Image Generation (2024.emnlp-main)

Copied to clipboard

Challenge: Existing evaluation benchmarks do not support arbitrarily interleaved images and text for both inputs and outputs.
Approach: They propose to use a benchmark to evaluate interleaved text-and-image generation . they define five evaluation aspects for InterleavatedEval, a reference-free metric .
Outcome: The proposed benchmarks cover a limited number of domains and use cases and lack comparableity-based metrics.
EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association (2025.acl-long)

Copied to clipboard

Challenge: Goal-oriented script planning is used by humans to plan for typical activities . however, this capability remains underexplored due to several challenges .
Approach: They propose a framework that enables product-enriched scripts by associating products with each step based on the semantic similarity between the actions and their purchase intentions.
Outcome: The proposed framework can generate product-enriched scripts from 2.4 million scripts . human annotations are conducted to provide gold labels for a sampled subset .
TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph (2021.emnlp-main)

Copied to clipboard

Challenge: Existing models infer the answer by predicting the sequential relation path or aggregating the hidden graph features.
Approach: They propose a model which jumps between entities at multiple steps . they demonstrate that TransferNet surpasses state-of-the-art models by a large margin .
Outcome: The proposed model surpasses state-of-the-art models on MetaQA and on other datasets.
GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solving (2024.findings-acl)

Copied to clipboard

Challenge: Recent advances in large language models (LLMs) and multi-modal models (MMs) have demonstrated remarkable capabilities in problem-solving, but their proficiency in tackling geometry math problems has not been thoroughly evaluated.
Approach: They propose a benchmark to evaluate the performance of large language models and multi-modal models in solving geometry math problems.
Outcome: The proposed model achieves 55.67% accuracy on main subset but only 6.00% accuracy on hard subset.
R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation (2025.emnlp-main)

Copied to clipboard

Challenge: Reasoning is a fundamental capability underpinning text-to-image (T2I) generation.
Approach: They propose a benchmark to rigorously assess reasoning-driven T2I generation.
Outcome: Experiments with 16 representative T2I models show limited reasoning performance . a strong pipeline-based framework decouples reasoning and generation .
Infinite Babble: Inflating 3D Vision-Language Model Inference Overhead via Adversarial Geometric Perturbation (2026.findings-acl)

Copied to clipboard

Challenge: 3D Vision-Language Models (VLMs) are critical cognitive backbone for spatial intelligence, but their reliance on autoregressive decoding introduces a fundamental vulnerability regarding inference efficiency.
Approach: They propose a framework that triggers computational and economic exhaustion in 3D-VLMs by injecting imperceptible noise that forces the model into a state of pathological verbosity.
Outcome: The proposed framework amplifies output length and energy consumption by up to 6.45, demonstrating a potent capability to drain system resources.
Benchmarking for Domain-Specific LLMs: A Case Study on Academia and Beyond (2025.findings-emnlp)

Copied to clipboard

Challenge: Comp-Comp is an iterative benchmarking framework grounded in the principles of comprehensiveness and compactness.
Approach: They propose a benchmark framework that incorporates the principle of comprehensiveness and compactness.
Outcome: The proposed framework is domain-agnostic and adaptable to a wide range of specialized fields.
Towards Statistical Factuality Guarantee for Large Vision-Language Models (2025.emnlp-main)

Copied to clipboard

Challenge: Advancements in Large Vision-Language Models (LVLMs) have demonstrated impressive performance in image-conditioned text generation, but hallucinated outputs pose a major barrier to their use in safety-critical applications.
Approach: They propose a conformal-prediction-based framework that achieves finite-sample distribution-free statistical guarantees to the factuality of LVLM output.
Outcome: The proposed framework reduces the error rate of LLaVa-1.5 claims from 87.8% to 10.0% while ensuring that the output is accurate.
SPUQ: Perturbation-Based Uncertainty Quantification for Large Language Models (2024.eacl-long)

Copied to clipboard

Challenge: Large language models have a tendency to make confidently wrong predictions, highlighting the need for uncertainty quantification (UQ) . previous studies focused on aleatoric uncertainty, but the full spectrum of uncertainties, including epistemic, remains inadequately explored.
Approach: They propose a method to quantify uncertainty in large language models (LLMs) they use a set of perturbations and an aggregation module to generalize the method.
Outcome: The proposed method improves model uncertainty calibration and reduces expected calibration error by 50% on average.
Unveiling the Implicit Toxicity in Large Language Models (2023.emnlp-main)

Copied to clipboard

Challenge: Recent studies focus on probing toxic outputs that can be easily detected with existing toxicity classifiers, but LLMs can generate diverse implicit toxic output that are difficult to detect via simply zero-shot prompting.
Approach: They propose a reinforcement learning based attacking method to induce the implicit toxic outputs in large language models by fine-tuning toxicity classifiers.
Outcome: The proposed method generates implicit toxic outputs that are difficult to detect via zero-shot prompting on five widely-adopted toxicity classifiers.
AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversation (2023.findings-acl)

Copied to clipboard

Challenge: Crowdsourced dialogue corpora are limited in scale and topic coverage due to the expensive cost of data curation.
Approach: They construct an augmented dataset for the emotional support conversation task using large language models for dialogue augmentation.
Outcome: The proposed approach outperforms baselines of dialogue augmentation and improves the model's generalization ability to open-domain topics.
SEE: Strategic Exploration and Exploitation for Cohesive In-Context Prompt Optimization (2025.acl-long)

Copied to clipboard

Challenge: Existing approaches separate the optimization of prompt instructions and in-context learning examples, leading to incohesive, suboptimal results.
Approach: They propose a framework that refines both prompt instructions and in-context learning examples.
Outcome: The proposed framework outperforms state-of-the-art prompt optimization methods on 35 benchmark tasks.
Multilingual Federated Low-Rank Adaptation for Collaborative Content Anomaly Detection across Multilingual Social Media Participants (2025.emnlp-main)

Copied to clipboard

Challenge: Recent developments in multilingual social media platforms (SNS) exacerbate new challenges in SNS content anomaly detection due to data islands and linguistic imbalance.
Approach: They propose a multilingual Federated LoRA based on SVD-based language-specific disentanglement of LoRA blocks and a local orthogonal tuning strategy to detect content anomalies.
Outcome: The proposed solution is superior in multilingual content anomaly detection while reducing multilingual knowledge conflicts and communication rounds.
T2R-BENCH: A Benchmark for Real World Table-to-Report Task (2025.emnlp-main)

Copied to clipboard

Challenge: Existing table benchmarks lack the capacity to adequately assess the practical application of table reasoning in industrial applications.
Approach: They propose a bilingual table-to-report task and a table-based benchmark to assess the quality of table reasoning.
Outcome: The proposed task is based on a bilingual benchmark with 457 industrial tables and evaluation criteria to measure the quality of report generation.
Persona-Guided Planning for Controlling the Protagonist’s Persona in Story Generation (2022.naacl-main)

Copied to clipboard

Challenge: Existing methods to control the protagonist's persona in story generation are implicitly and sparsely embodied in stories, so we propose a planning-based generation model called ConPer to explicitly model the relationship between personas and events.
Approach: They propose a model to control the protagonist's persona in story generation by predicting one target sentence and planning the plot as a sequence of keywords with the guidance of the predicted persona-related events and commonsense knowledge.
Outcome: The proposed model outperforms state-of-the-art models for generating more coherent and persona-controllable stories.
The GaoYao Benchmark: A Comprehensive Framework for Evaluating Multilingual and Multicultural Abilities of Large Language Models (2026.acl-long)

Copied to clipboard

Challenge: Existing multilingual evaluation benchmarks neglect cultural nuances and lack language coverage in subjective tasks.
Approach: They propose a framework that categorizes evaluation tasks into three cultural layers and nine cognitive sub-layers.
Outcome: The proposed framework surpasses prior coverage by up to 111% on 20+ LLMs.
KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base (2022.acl-long)

Copied to clipboard

Challenge: Existing benchmarks for Complex KBQA lack compositional reasoning capabilities . Existing methods for Complex questions are poor in diversity or scale .
Approach: They propose a compositional programming language to represent the reasoning process of complex questions.
Outcome: The proposed dataset includes around 120K diverse natural language questions . it provides a compositional and interpretable programming language to represent the reasoning process of complex questions based on the proposed model .
GOLD: Geometry Problem Solver with Natural Language Description (2024.findings-naacl)

Copied to clipboard

Challenge: Existing methods for solving geometry math problems struggle with accurately interpreting geometry diagrams, posing a challenge for problem-solving.
Approach: They propose a model that extracts geometric relations from diagrams and converts them into natural language descriptions.
Outcome: The proposed model outperforms the previous best method on the UniGeo dataset by 12.7% and 42.1% in calculation and proving subsets.
MatchPrompt: Prompt-based Open Relation Extraction with Semantic Consistency Guided Clustering (2022.emnlp-main)

Copied to clipboard

Challenge: Existing methods for open relation extraction (OpenRE) focus on labeled and pre-defined instances, which are costly to acquire in reality.
Approach: They propose a framework that can extract relations without pre-defined types from open-domain corpus with efficient knowledge transfer from a few pre-determined relational instances.
Outcome: The proposed framework achieves the new SOTA results for OpenRE on different datasets.
Survival of the Safest: Towards Secure Prompt Optimization through Interleaved Multi-Objective Evolution (2024.emnlp-industry)

Copied to clipboard

Challenge: Experimental evaluations across diverse benchmark datasets affirm ‘s efficacy in delivering high performance and notably enhancing safety and security compared to single-objective methods.
Approach: They propose a multi-objective prompt optimization framework that integrates semantic, feedback, and crossover mutations to effectively traverse the prompt landscape.
Outcome: The proposed framework enhances both performance and security in large language models while keeping computational demands low.
RedundancyLens: Revealing and Exploiting Visual Token Processing Redundancy for Efficient Decoder-Only MLLMs (2025.findings-acl)

Copied to clipboard

Challenge: Current decoder-only architectures achieve higher performance but lower efficiency . cross-attention-based architectures skip visual token computations .
Approach: They propose a training-free framework for analyzing trained MLLMs to investigate redundancy . they propose 'probe-activated Dynamic FFN and Hollow Attention' algorithms for visual token reductions and a layer ranking algorithm for inference acceleration.
Outcome: The proposed framework achieves comparable performance to or better than state-of-the-art methods while remaining compatible with them.
Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation (2024.emnlp-main)

Copied to clipboard

Challenge: Language models suffer from poor interpretability and transparency, as well as the intrinsic risk of hallucination and misinformation.
Approach: They propose a statistical framework that assesses how well a query can be answered by an RAG system by capturing the relevance of knowledge.
Outcome: The proposed framework assesses how well a query can be answered by an RAG system by capturing the relevance of knowledge.
OneRec-Think: In-Text Reasoning for Generative Recommendation (2026.acl-long)

Copied to clipboard

Challenge: Existing generative models lack the capacity for explicit and controllable reasoning, a key advantage of LLMs.
Approach: They propose a framework that integrates dialogue, reasoning, and personalized recommendation.
Outcome: Experiments across public benchmarks show state-of-the-art performance.
Causally Modeling the Linguistic and Social Factors that Predict Email Response (2025.naacl-long)

Copied to clipboard

Challenge: a key intent behind many emails is to get a reply from the recipient.
Approach: They propose to model the intents, expectations, and responsiveness in email exchanges by using a dataset containing 1800 emails annotated with nuanced types of intents and expectations.
Outcome: The proposed model is based on 1800 emails annotated with nuanced types of intents and expectations . it shows that social status, argumentation, and strength of social connection influence email response rates .
AutoCAD: Automatically Generate Counterfactuals for Mitigating Shortcut Learning (2022.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for generating counterfactuals rely on human efforts or task-specific designs.
Approach: They propose to use a fully automatic and task-agnostic CAD generation framework to generate diverse counterfactuals.
Outcome: The proposed framework outperforms human-in-the-loop and task-specific CAD methods on multiple out-of-domain and challenge benchmarks.
Effectiveness of Pre-training for Few-shot Intent Classification (2021.findings-emnlp)

Copied to clipboard

Challenge: Existing paradigms further pre-train language models such as BERT on vast amount of unlabeled corpus, but we find it highly effective and efficient to simply fine-tune BERT with roughly 1,000 labeled utterances from public datasets.
Approach: They propose to fine-tune BERT with a small set of labeled utterances from public datasets to achieve a pre-trained model based on a set of 1,000 labeles.
Outcome: The proposed model can outperform existing models on domains with very different semantics on novel domains.
Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training (2021.emnlp-main)

Copied to clipboard

Challenge: Named entity recognition models require abundant high-quality annotations to train . distant supervision may induce incomplete and noisy labels, making supervised learning ineffective.
Approach: They propose a noise-robust learning scheme for training named entity recognition models using only distantly-labeled data and a self-training method that uses contextualized augmentations created by pre-trained language models.
Outcome: The proposed method outperforms existing supervised NER models on three datasets by significant margins.
Divide-Conquer-Reasoning for Consistency Evaluation and Automatic Improvement of Large Language Models (2024.emnlp-industry)

Copied to clipboard

Challenge: Existing methods for evaluating the quality and consistency of text generated by Large Language Models are not effective.
Approach: They propose a divide-conquer-reasoning approach to evaluate LLM-generated texts using a split-and-conquers evaluator and an automatic metric converter to facilitate this approach.
Outcome: The proposed framework outperforms state-of-the-art methods by a large margin on multiple benchmarks and reduces 90% of output inconsistencies in one iteration.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations