Papers by Jiawei Zhou

47 papers
HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference (2025.findings-acl)

Copied to clipboard

Challenge: Existing top-k attention methods struggle to strike a balance between efficiency and accuracy.
Approach: They propose a top-k attention approach that integrates low-overhead techniques into the Top-k Attention process to achieve 7.2 speedup compared to vanilla full attention.
Outcome: The proposed approach achieves 7.2 speedup compared to current top-k attention methods while maintaining model accuracy.
Flames: Benchmarking Value Alignment of LLMs in Chinese (2024.naacl-long)

Copied to clipboard

Challenge: Existing benchmarks for large language models (LLMs) do not accurately uncover safety vulnerabilities in LLMs.
Approach: They propose a value alignment benchmark called Flames that encompasses both harmlessness principles and a unique morality dimension that integrates specific Chinese values such as harmony.
Outcome: The proposed model performs poorly on Flames, particularly in safety and fairness dimensions.
Grasping the Essentials: Tailoring Large Language Models for Zero-Shot Relation Extraction (2024.emnlp-main)

Copied to clipboard

Challenge: Existing Relation extraction models require extensive annotated training data, which is costly and labor-intensive to collect.
Approach: They propose a new zero-shot RE task where only relation definitions are provided instead of seen-unseen relation instances.
Outcome: The proposed task significantly improves cost-effective zero-shot performance by large margins.
CODE-MVP: Learning to Represent Source Code from Multiple Views with Contrastive Pre-Training (2022.findings-naacl)

Copied to clipboard

Challenge: Recent studies have focused on code representation learning, which aims to represent the semantics of source code into distributed vectors.
Approach: They propose to integrate different views with the natural-language description of source code into a unified framework with Multi-View contrastive Pre-training.
Outcome: The proposed model outperforms state-of-the-art models on three downstream tasks over five datasets.
MART: Improving LLM Safety with Multi-round Automatic Red-Teaming (2024.naacl-long)

Copied to clipboard

Challenge: Existing red-teaming methods for large language models often discover safety risks without addressing them.
Approach: They propose a multi-round automatic red-teaming method that incorporates both adversarial prompt writing and safe response generation.
Outcome: The proposed method significantly increases red-teaming scalability and the safety of the target LLM.
Tracking the Limits of Knowledge Propagation: How LLMs Fail at Multi-Step Reasoning with Conflicting Knowledge (2026.eacl-long)

Copied to clipboard

Challenge: Existing benchmarks for analyzing the performance of Large Language Models (LLMs) focus on single knowledge updates and fact recall, but do not consider how these updates affect downstream reasoning.
Approach: They propose a benchmark to study how LLMs propagate new knowledge when it conflicts with the model's parametric knowledge.
Outcome: The proposed benchmark compared models with no updated facts to show that the new methods worsen performance and improve reasoning performance.
ReCreate: Reasoning and Creating Domain Agents Driven by Experience (2026.acl-long)

Copied to clipboard

Challenge: Large Language Model (LLM) agents are reshaping the industrial landscape, but tasks differ widely, making them labor-intensive to build.
Approach: They propose an experience-driven framework for the automatic creation of domain agents . they leverage agent interaction histories to provide rich concrete signals on success or failure .
Outcome: The proposed framework outperforms human-designed agents and existing methods in experiments across diverse domains.
AssistedDS: Benchmarking How External Domain Knowledge Assists LLMs in Automated Data Science (2025.findings-emnlp)

Copied to clipboard

Challenge: Large language models (LLMs) have advanced the automation of data science workflows, yet it remains unclear whether they can critically leverage external domain knowledge as human data scientists do in practice.
Approach: They propose a benchmark to evaluate how large language models handle external domain knowledge in tabular prediction tasks.
Outcome: The proposed model evaluates whether it can critically leverage external domain knowledge as human data scientists do in practice.
Reaction Miner: An Integrated System for Chemical Reaction Extraction from Textual Data (2023.emnlp-demo)

Copied to clipboard

Challenge: Reaction Miner is a system designed to extract chemical reactions from raw scientific PDFs.
Approach: They propose a system that extracts chemical reactions directly from raw scientific PDFs.
Outcome: The proposed system can extract chemical reactions from raw scientific PDFs.
Conditional Dichotomy Quantification via Geometric Embedding (2025.acl-long)

Copied to clipboard

Challenge: Existing methods that rely on semantic similarity fail to capture the nuanced oppositional dynamics essential for these applications.
Approach: They propose a task that formalizes the measurement of conditional dichotomy by using a dichotomian framework.
Outcome: The proposed framework provides carefully constructed datasets covering debate, defeasible inference, and causal reasoning scenarios.
ConsistentChat: Building Skeleton-Guided Consistent Multi-Turn Dialogues for Large Language Models from Scratch (2025.emnlp-main)

Copied to clipboard

Challenge: Existing instruction data synthesis methods focus on single-turn instructions and neglect cross-turn coherence, resulting in context drift and reduced task completion rates.
Approach: They propose a framework that constrains multi-turn instruction synthesis by explicitly modeling human conversational intent.
Outcome: The proposed framework outperforms existing models trained on single-turn and multi-turn instruction datasets.
Online Semantic Parsing for Latency Reduction in Task-Oriented Dialogue (2022.acl-long)

Copied to clipboard

Challenge: Standard conversational semantic parsing maps a user's intent into an executable program, but execution is slow when expensive function calls are included.
Approach: They propose a task of online semantic parsing to predict and execute function calls while the user is still speaking.
Outcome: The proposed approach reduces latency with good parsing quality and execution cost.
VPL: Visual Proxy Learning Framework for Zero-Shot Medical Image Diagnosis (2024.findings-emnlp)

Copied to clipboard

Challenge: Insufficient medical text precision and the modal disparity between text and vision spaces pose challenges for vision-language models like CLIP.
Approach: They propose a visual proxy learning framework that combines a text refinement module and a stable Sinkhorn algorithm to enhance the diagnostic performance.
Outcome: The proposed model outperforms the state-of-the-art CLIP inference by 1.69% to 15.31% on five datasets covering various diseases.
Catch Me If You Can? Not Yet: LLMs Still Struggle to Imitate the Implicit Writing Styles of Everyday Authors (2025.findings-emnlp)

Copied to clipboard

Challenge: Personal style is often subtle and implicit, making it difficult to specify through prompts yet essential for user-aligned generation.
Approach: They evaluate LLMs' ability to imitate personal writing styles via in-context learning from user-authored samples.
Outcome: The proposed model can imitate personal writing styles from a small number of user-authored samples.
HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token (2026.eacl-long)

Copied to clipboard

Challenge: Existing methods for detection of hallucinations operate after text generation, making intervention costly and untimely.
Approach: They examine whether hallucination risk can instead be predicted before any token is generated by probing a model's internal representations in a single forward pass.
Outcome: The proposed model can detect hallucinations before token generation, while query-token representations can be more accurate.
Gender Inclusivity Fairness Index (GIFI): A Multilevel Framework for Evaluating Gender Diversity in Large Language Models (2025.acl-long)

Copied to clipboard

Challenge: GIFI measures the diversity of LLMs' outputs, including gender identifiers, and identifies gender biases associated with varying gender identifiers.
Approach: They propose a Gender Inclusivity Fairness Index (GIFI) that quantifies the diverse gender inclusivity of large language models.
Outcome: The proposed metric quantifies the diversity of LLMs across multiple dimensions, including non-binary identities.
Unraveling Misinformation Propagation in LLM Reasoning (2025.findings-emnlp)

Copied to clipboard

Challenge: Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning, but how they propagate within their reasoning process remains underexplored.
Approach: They propose a practical approach to mitigating misinformation propagation in LLMs by applying factual corrections early in the reasoning process and fine-tuning on synthesized data with early-stage corrections significantly improves reasoning factuality.
Outcome: The proposed model can correct misinformation when explicitly instructed, but fails to correct misinformation less than half the time even with explicit instructions.
Reverse Modeling in Large Language Models (2025.naacl-short)

Copied to clipboard

Challenge: Using pre-trained LLMs with reversed text inputs can improve their performance across multiple languages.
Approach: They propose a way to determine whether LLMs can understand reversed text inputs by reversing entire paragraphs or documents at the token level.
Outcome: The proposed model can be used to improve understanding across multiple languages.
Topic-Oriented Open Relation Extraction with A Priori Seed Generation (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods for open relation extraction give sub-optimal results on specific topics.
Approach: They propose a method that leverages the built-in knowledge of large language models to maintain a dynamic seed relation dictionary for the topic.
Outcome: The proposed approach empowers better topic-oriented control over the generated relations and improves ORE performance along the five dimensions, especially on specialized and narrow topics.
CROP: Contextual Region-Oriented Visual Token Pruning (2025.emnlp-main)

Copied to clipboard

Challenge: Existing VLMs process entire images, leading to excessive visual tokens . redundant image information also introduces a large number of visual token, requiring much higher memory and computation in VLM.
Approach: They propose a framework to prune visual tokens using localization and pruning . they propose CROP to locate local image regions relevant to the query .
Outcome: The proposed framework outperforms existing visual token pruning methods on a wide range of tasks.
RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System (2021.naacl-demos)

Copied to clipboard

Challenge: We present a new information extraction system that can construct temporal event graphs from news documents.
Approach: They propose a temporal event graph extraction system that can extract news documents . they extend the system from sentence-level event extraction to cross-document cross-media event extraction .
Outcome: The proposed system can extract temporal event graphs from news documents in multiple languages and multiple data modalities.
ARK: Answer-Centric Retriever Tuning via KG-augmented Curriculum Learning (2026.acl-long)

Copied to clipboard

Challenge: Retrieval-Augmented Generation (RAG) is a powerful framework for knowledge-intensive tasks, but its effectiveness in long-context scenarios is often bottlenecked by the retriever’s inability to distinguish sparse yet crucial evidence.
Approach: They propose a framework that fine-tunes the retriever for Answer Alignment by identifying high-quality positive chunks by evaluating their sufficiency to generate the correct answer.
Outcome: The proposed framework improves 14.5% over the base model and maintains strong efficiency for long-context RAG.
Context-Efficient Retrieval with Factual Decomposition (2025.naacl-short)

Copied to clipboard

Challenge: Existing models that use dynamically expanding text can be incorporated into large language models.
Approach: They show that pre-processing external corpus into semi-structured "atomic facts" reduces the size of the context and improves inference efficiency.
Outcome: The proposed form of atomic facts improves on question answering tasks when the amount of retrieved text is limited.
Fundamental Capabilities of Large Language Models and their Applications in Domain Scenarios: A Survey (2024.acl-long)

Copied to clipboard

Challenge: Existing benchmark-based evaluations cannot accurately reflect the performance of real-world applications.
Approach: They propose a reliable strategy for domains to choose more robust LLMs for real-world applications.
Outcome: The proposed strategy addresses the challenges faced by domains to choose more robust LLMs for real-world applications.
T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback (2025.acl-long)

Copied to clipboard

Challenge: Text-to-audio (T2A) models still struggle to satisfy human preferences for prompt-following and acoustic quality when generating complex multi-event audio.
Approach: They propose to use AI feedback learning to enhance basic capabilities of text-to-audio models . they use a large audio preference dataset to evaluate the model's capabilities .
Outcome: The proposed model improves in simple and complex scenarios with AI feedback learning.
Quick Back-Translation for Unsupervised Machine Translation (2023.findings-emnlp)

Copied to clipboard

Challenge: Unsupervised machine translation models are limited by the run-time of autoregressive inference during back-translation and lack of synthetic data efficiency.
Approach: They propose a two-for-one improvement to Transformer back-translation: Quick Back-Translation (QBT). QBT re-purposes the encoder as a generative model, and uses encoder-generated sequences to train the decoder.
Outcome: Experiments on various WMT benchmarks show that QBT dramatically outperforms standard back-translation only method in terms of training efficiency for comparable translation qualities.
Agent Trading Arena: A Study on Numerical Understanding in LLM-Based Agents (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches to large language models are limited to historical backtesting and static data.
Approach: a new large-language model is developed to simulate real-time trading in a virtual stock market . the agent trading arena simulates real-world bid-ask interactions and provides real-life trading scenarios .
Outcome: The Agent Trading Arena simulates real-world market conditions and directly impacts price dynamics.
Simple Unsupervised Summarization by Contextual Matching (P19-1)

Copied to clipboard

Challenge: Existing methods for sentence summarization require a large amount of parallel data for supervision to work.
Approach: They propose an unsupervised method for sentence summarization using only language modeling.
Outcome: The proposed method maintains continuous contextual matching while maintaining output fluency without any paired examples.
Omni-I2C: A Holistic Benchmark for High-Fidelity Image-to-Code Generation (2026.acl-long)

Copied to clipboard

Challenge: a benchmark is designed to evaluate the capability of Large Multimodal Models (LMMs) in converting complex, structured digital graphics into executable code.
Approach: They propose a benchmark to evaluate the capability of Large Multimodal Models to convert digital graphics into executable code.
Outcome: The proposed benchmark exposes the performance gap among leading LMMs . the benchmark features 1130 meticulously curated samples .
When Rules Learn: A Self-Evolving Agent for Legal Case Retrieval (2026.findings-acl)

Copied to clipboard

Challenge: Existing dense retrieval methods have achieved notable progress, but their effectiveness in legal case retrieval remains limited.
Approach: They propose a self-evolving framework for rule-driven query rewriting that enhances BM25 without any parameter training.
Outcome: The proposed framework outperforms non-evolutionary baselines, including human-designed rules and greedy rule selection, especially when powered by a high-capacity core LLM.
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)

Copied to clipboard

Challenge: Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work.
Approach: They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations.
Outcome: The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work.
From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens (2025.emnlp-demos)

Copied to clipboard

Challenge: Existing vision-language models are based on exactmatch based accuracy and its derivations to evaluate performance.
Approach: They propose a toolkit that supports systematic benchmarking, analysis, and interpretation of vision-language models by extracting intermediate outputs from any layer during the forward pass of open-source VLMs.
Outcome: The proposed toolkit supports 16 state-of-the-art base VLMs and their over 30 variants and is extensible to accommodate new models without changing the core logic.
Towards Open Environment Intent Prediction (2023.findings-acl)

Copied to clipboard

Challenge: Out-of-Domain (OOD) Intent Classification and New Intent Discovering are two tasks in the Task-Oriented Dialogue System.
Approach: They propose a task paradigm to extend Out-of-Domain (OOD) Intent Classification and New Intent Discovering tasks in the Task-Oriented Dialogue System.
Outcome: The proposed scheme improves on existing OOD intent classification and discovery datasets.
Text or Pixels? Evaluating Efficiency and Understanding of LLMs with Visual Text Inputs (2025.findings-emnlp)

Copied to clipboard

Challenge: *visual text representations* are a practical and surprisingly effective form of input compression for decoder LLMs.
Approach: They exploit visual representations to render long text inputs as a single image and provide it directly to the model.
Outcome: The proposed method reduces token usage while preserving performance.
PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models (2025.acl-long)

Copied to clipboard

Challenge: Recent large language models (LLMs) have achieved significant performance in complex reasoning tasks such as mathematics and code generation.
Approach: They propose a process-level benchmark specifically designed to assess the fine-grained error detection capabilities of PRMs.
Outcome: The proposed model measures the accuracy, soundness, and sensitivity of 25 models across open-source and closed-source large language models.
AMR Parsing with Action-Pointer Transformer (2021.naacl-main)

Copied to clipboard

Challenge: Abstract Meaning Representation parsing is a sentence-to-graph prediction task . graph nodes are semantically based on one or more sentence tokens, so implicit alignments can be derived.
Approach: They propose a transition-based system that decouples hard-attention over sentences with a target-side action pointer mechanism to decouple source tokens from node representations and address alignments.
Outcome: The proposed system achieves the second best Smatch score on AMR 2.0 (81.8) it decouples source tokens from node representations and addresses alignments, but lacks expressiveness.
Text2DB: Integration-Aware Information Extraction with Large Language Model Agents (2024.findings-acl)

Copied to clipboard

Challenge: Current methods for information extraction (IE) focus on integrating IE output with the database . a long-overlooked question is what counts as "relevant knowledge"
Approach: They propose a task that emphasizes integration of IE output and the database . they introduce a benchmark and an LLM agent framework for this task .
Outcome: The proposed task integrates IE output and the target database (or knowledge base) it meets common demands such as data infilling, row population, and column addition .
Structure-aware Fine-tuning of Sequence-to-sequence Transformers for Transition-based AMR Parsing (2021.emnlp-main)

Copied to clipboard

Challenge: Recent work shows that pre-trained sequence-to-sequence Transformer models are effective in predicting linearized Abstract Meaning Representation graphs.
Approach: They propose a structure-aware transition-based approach to AMR parsing that integrates general pre-trained sequence-to-sequence language models with a structured transition set.
Outcome: The proposed approach retains the desirable properties of previous approaches while reaching the new parsing state of the art for AMR 2.0.
MedCoT: Medical Chain of Thought via Hierarchical Expert (2024.emnlp-main)

Copied to clipboard

Challenge: Existing methods for medical visual question answering lack robustness and reasoning paths for real-world medical diagnostics.
Approach: They propose a hierarchical expert verification reasoning chain method to enhance interpretability and accuracy in medical visual question answering.
Outcome: The proposed method outperforms existing methods on four standard Med-VQA datasets.
Improving Non-autoregressive Neural Machine Translation with Monolingual Data (2020.acl-main)

Copied to clipboard

Challenge: Neural machine translation is usually done via knowledge distillation from an autoregressive (AR) model.
Approach: They leverage large monolingual corpora to improve the NAR model's performance by transferring the autoregressive model' s generalization ability while preventing overfitting.
Outcome: The proposed methods on the WMT14 En-De and WMT16 En-Ro news translation tasks show that monolingual data augmentation improves the NAR model to approach the teacher AR model’s performance.
A Community-Centric Perspective for Characterizing and Detecting Anti-Asian Violence-Provoking Speech (2024.acl-long)

Copied to clipboard

Challenge: Using data from 420k Twitter posts, we characterize anti-Asian violence-provoking speech and collect a community-crowdsourced dataset to facilitate its large-scale detection.
Approach: They develop a codebook to characterize anti-Asian violence-provoking speech and collect a community-crowdsourced dataset to facilitate its large-scale detection.
Outcome: The proposed codebook analyzes 420k tweets over 3 years and compares classifiers with hateful speech classifier classifier to detect hateful content.
HyperMem: Hypergraph Memory for Long-Term Conversations (2026.acl-long)

Copied to clipboard

Challenge: Existing approaches to long-term memory management rely on pairwise relations, causing fragmented retrieval.
Approach: They propose a hypergraph-based hierarchical memory architecture that explicitly models high-order associations using hyperedges.
Outcome: Experiments show that HyperMem achieves state-of-the-art performance with 92.73% accuracy for long-term conversations.
Frame First, Then Extract: A Frame-Semantic Reasoning Pipeline for Zero-Shot Relation Triplet Extraction (2025.emnlp-main)

Copied to clipboard

Challenge: Existing methods to extract triplets for unseen relations rely on costly fine-tuning and lack structured semantic guidance.
Approach: They propose a framework that adopts a "frame first, then extract" paradigm to extract triplets from unstructured text.
Outcome: The proposed framework achieves competitive zero-shot performance on multiple benchmarks and can be used to enhance existing extraction methods.
Inducing and Using Alignments for Transition-based AMR Parsing (2022.naacl-main)

Copied to clipboard

Challenge: Abstract Meaning Representation parsers rely on node-to-word alignments, but lack the complexity of the pipeline.
Approach: They propose a neural aligner for abstract meaning representation that learns node-to-word alignments without relying on pipelines.
Outcome: The proposed approach improves accuracy and generalization from AMR2.0 to AMR3.0 corpora.
Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answering (2022.acl-long)

Copied to clipboard

Challenge: Existing methods to train dense passage retrieval have a large data gap between upstream and downstream relevance.
Approach: They propose a method to pre-train the dense retriever with the text relevance induced by hyperlinks within Web documents.
Outcome: The proposed method outperforms existing methods under different scenarios and in the open-domain question answering domain.
Re3: Relevance & Recency Retrieval for Mitigating Temporal Hallucination (2026.acl-long)

Copied to clipboard

Challenge: Existing retrievers suffer from temporal-semantic misalignment and outdated-document interference . Existing frameworks suffer from both temporal validity and outdated factual versions .
Approach: They propose a framework that mitigates temporal hallucinations by embedding heterogeneous temporal signals into the semantic space to ensure retrieval fidelity.
Outcome: Experiments show that Re3 outperforms baselines by 9.7% in generation accuracy . the framework outperformed strongest baselines on challenging dynamic tasks .

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations