Papers by Jiawei Zhou
Copied to clipboard
| Challenge: | Existing top-k attention methods struggle to strike a balance between efficiency and accuracy. |
| Approach: | They propose a top-k attention approach that integrates low-overhead techniques into the Top-k Attention process to achieve 7.2 speedup compared to vanilla full attention. |
| Outcome: | The proposed approach achieves 7.2 speedup compared to current top-k attention methods while maintaining model accuracy. |
Copied to clipboard
| Challenge: | Existing benchmarks for large language models (LLMs) do not accurately uncover safety vulnerabilities in LLMs. |
| Approach: | They propose a value alignment benchmark called Flames that encompasses both harmlessness principles and a unique morality dimension that integrates specific Chinese values such as harmony. |
| Outcome: | The proposed model performs poorly on Flames, particularly in safety and fairness dimensions. |
Copied to clipboard
| Challenge: | Existing Relation extraction models require extensive annotated training data, which is costly and labor-intensive to collect. |
| Approach: | They propose a new zero-shot RE task where only relation definitions are provided instead of seen-unseen relation instances. |
| Outcome: | The proposed task significantly improves cost-effective zero-shot performance by large margins. |
Copied to clipboard
| Challenge: | Recent studies have focused on code representation learning, which aims to represent the semantics of source code into distributed vectors. |
| Approach: | They propose to integrate different views with the natural-language description of source code into a unified framework with Multi-View contrastive Pre-training. |
| Outcome: | The proposed model outperforms state-of-the-art models on three downstream tasks over five datasets. |
Copied to clipboard
| Challenge: | Existing red-teaming methods for large language models often discover safety risks without addressing them. |
| Approach: | They propose a multi-round automatic red-teaming method that incorporates both adversarial prompt writing and safe response generation. |
| Outcome: | The proposed method significantly increases red-teaming scalability and the safety of the target LLM. |
Copied to clipboard
| Challenge: | Existing benchmarks for analyzing the performance of Large Language Models (LLMs) focus on single knowledge updates and fact recall, but do not consider how these updates affect downstream reasoning. |
| Approach: | They propose a benchmark to study how LLMs propagate new knowledge when it conflicts with the model's parametric knowledge. |
| Outcome: | The proposed benchmark compared models with no updated facts to show that the new methods worsen performance and improve reasoning performance. |
Copied to clipboard
| Challenge: | Large Language Model (LLM) agents are reshaping the industrial landscape, but tasks differ widely, making them labor-intensive to build. |
| Approach: | They propose an experience-driven framework for the automatic creation of domain agents . they leverage agent interaction histories to provide rich concrete signals on success or failure . |
| Outcome: | The proposed framework outperforms human-designed agents and existing methods in experiments across diverse domains. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced the automation of data science workflows, yet it remains unclear whether they can critically leverage external domain knowledge as human data scientists do in practice. |
| Approach: | They propose a benchmark to evaluate how large language models handle external domain knowledge in tabular prediction tasks. |
| Outcome: | The proposed model evaluates whether it can critically leverage external domain knowledge as human data scientists do in practice. |
Copied to clipboard
| Challenge: | Reaction Miner is a system designed to extract chemical reactions from raw scientific PDFs. |
| Approach: | They propose a system that extracts chemical reactions directly from raw scientific PDFs. |
| Outcome: | The proposed system can extract chemical reactions from raw scientific PDFs. |
Copied to clipboard
| Challenge: | Existing methods that rely on semantic similarity fail to capture the nuanced oppositional dynamics essential for these applications. |
| Approach: | They propose a task that formalizes the measurement of conditional dichotomy by using a dichotomian framework. |
| Outcome: | The proposed framework provides carefully constructed datasets covering debate, defeasible inference, and causal reasoning scenarios. |
Copied to clipboard
| Challenge: | Existing instruction data synthesis methods focus on single-turn instructions and neglect cross-turn coherence, resulting in context drift and reduced task completion rates. |
| Approach: | They propose a framework that constrains multi-turn instruction synthesis by explicitly modeling human conversational intent. |
| Outcome: | The proposed framework outperforms existing models trained on single-turn and multi-turn instruction datasets. |
Copied to clipboard
| Challenge: | Standard conversational semantic parsing maps a user's intent into an executable program, but execution is slow when expensive function calls are included. |
| Approach: | They propose a task of online semantic parsing to predict and execute function calls while the user is still speaking. |
| Outcome: | The proposed approach reduces latency with good parsing quality and execution cost. |
Copied to clipboard
| Challenge: | Insufficient medical text precision and the modal disparity between text and vision spaces pose challenges for vision-language models like CLIP. |
| Approach: | They propose a visual proxy learning framework that combines a text refinement module and a stable Sinkhorn algorithm to enhance the diagnostic performance. |
| Outcome: | The proposed model outperforms the state-of-the-art CLIP inference by 1.69% to 15.31% on five datasets covering various diseases. |
Copied to clipboard
| Challenge: | Personal style is often subtle and implicit, making it difficult to specify through prompts yet essential for user-aligned generation. |
| Approach: | They evaluate LLMs' ability to imitate personal writing styles via in-context learning from user-authored samples. |
| Outcome: | The proposed model can imitate personal writing styles from a small number of user-authored samples. |
Copied to clipboard
| Challenge: | Existing methods for detection of hallucinations operate after text generation, making intervention costly and untimely. |
| Approach: | They examine whether hallucination risk can instead be predicted before any token is generated by probing a model's internal representations in a single forward pass. |
| Outcome: | The proposed model can detect hallucinations before token generation, while query-token representations can be more accurate. |
Copied to clipboard
| Challenge: | GIFI measures the diversity of LLMs' outputs, including gender identifiers, and identifies gender biases associated with varying gender identifiers. |
| Approach: | They propose a Gender Inclusivity Fairness Index (GIFI) that quantifies the diverse gender inclusivity of large language models. |
| Outcome: | The proposed metric quantifies the diversity of LLMs across multiple dimensions, including non-binary identities. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning, but how they propagate within their reasoning process remains underexplored. |
| Approach: | They propose a practical approach to mitigating misinformation propagation in LLMs by applying factual corrections early in the reasoning process and fine-tuning on synthesized data with early-stage corrections significantly improves reasoning factuality. |
| Outcome: | The proposed model can correct misinformation when explicitly instructed, but fails to correct misinformation less than half the time even with explicit instructions. |
Copied to clipboard
| Challenge: | Using pre-trained LLMs with reversed text inputs can improve their performance across multiple languages. |
| Approach: | They propose a way to determine whether LLMs can understand reversed text inputs by reversing entire paragraphs or documents at the token level. |
| Outcome: | The proposed model can be used to improve understanding across multiple languages. |
Copied to clipboard
| Challenge: | Existing methods for open relation extraction give sub-optimal results on specific topics. |
| Approach: | They propose a method that leverages the built-in knowledge of large language models to maintain a dynamic seed relation dictionary for the topic. |
| Outcome: | The proposed approach empowers better topic-oriented control over the generated relations and improves ORE performance along the five dimensions, especially on specialized and narrow topics. |
Copied to clipboard
| Challenge: | Existing VLMs process entire images, leading to excessive visual tokens . redundant image information also introduces a large number of visual token, requiring much higher memory and computation in VLM. |
| Approach: | They propose a framework to prune visual tokens using localization and pruning . they propose CROP to locate local image regions relevant to the query . |
| Outcome: | The proposed framework outperforms existing visual token pruning methods on a wide range of tasks. |
Copied to clipboard
| Challenge: | We present a new information extraction system that can construct temporal event graphs from news documents. |
| Approach: | They propose a temporal event graph extraction system that can extract news documents . they extend the system from sentence-level event extraction to cross-document cross-media event extraction . |
| Outcome: | The proposed system can extract temporal event graphs from news documents in multiple languages and multiple data modalities. |
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) is a powerful framework for knowledge-intensive tasks, but its effectiveness in long-context scenarios is often bottlenecked by the retriever’s inability to distinguish sparse yet crucial evidence. |
| Approach: | They propose a framework that fine-tunes the retriever for Answer Alignment by identifying high-quality positive chunks by evaluating their sufficiency to generate the correct answer. |
| Outcome: | The proposed framework improves 14.5% over the base model and maintains strong efficiency for long-context RAG. |
Copied to clipboard
| Challenge: | Existing models that use dynamically expanding text can be incorporated into large language models. |
| Approach: | They show that pre-processing external corpus into semi-structured "atomic facts" reduces the size of the context and improves inference efficiency. |
| Outcome: | The proposed form of atomic facts improves on question answering tasks when the amount of retrieved text is limited. |
Copied to clipboard
| Challenge: | Existing methods for event prediction are incomplete and noisy. |
| Approach: | They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework . |
| Outcome: | The proposed framework can be applied to a wide variety of newsworthy scenarios. |
Copied to clipboard
| Challenge: | Existing benchmark-based evaluations cannot accurately reflect the performance of real-world applications. |
| Approach: | They propose a reliable strategy for domains to choose more robust LLMs for real-world applications. |
| Outcome: | The proposed strategy addresses the challenges faced by domains to choose more robust LLMs for real-world applications. |
Copied to clipboard
| Challenge: | Text-to-audio (T2A) models still struggle to satisfy human preferences for prompt-following and acoustic quality when generating complex multi-event audio. |
| Approach: | They propose to use AI feedback learning to enhance basic capabilities of text-to-audio models . they use a large audio preference dataset to evaluate the model's capabilities . |
| Outcome: | The proposed model improves in simple and complex scenarios with AI feedback learning. |
Copied to clipboard
| Challenge: | Unsupervised machine translation models are limited by the run-time of autoregressive inference during back-translation and lack of synthetic data efficiency. |
| Approach: | They propose a two-for-one improvement to Transformer back-translation: Quick Back-Translation (QBT). QBT re-purposes the encoder as a generative model, and uses encoder-generated sequences to train the decoder. |
| Outcome: | Experiments on various WMT benchmarks show that QBT dramatically outperforms standard back-translation only method in terms of training efficiency for comparable translation qualities. |
Copied to clipboard
| Challenge: | Existing approaches to large language models are limited to historical backtesting and static data. |
| Approach: | a new large-language model is developed to simulate real-time trading in a virtual stock market . the agent trading arena simulates real-world bid-ask interactions and provides real-life trading scenarios . |
| Outcome: | The Agent Trading Arena simulates real-world market conditions and directly impacts price dynamics. |
Copied to clipboard
| Challenge: | Existing methods for sentence summarization require a large amount of parallel data for supervision to work. |
| Approach: | They propose an unsupervised method for sentence summarization using only language modeling. |
| Outcome: | The proposed method maintains continuous contextual matching while maintaining output fluency without any paired examples. |
Copied to clipboard
| Challenge: | a benchmark is designed to evaluate the capability of Large Multimodal Models (LMMs) in converting complex, structured digital graphics into executable code. |
| Approach: | They propose a benchmark to evaluate the capability of Large Multimodal Models to convert digital graphics into executable code. |
| Outcome: | The proposed benchmark exposes the performance gap among leading LMMs . the benchmark features 1130 meticulously curated samples . |
Copied to clipboard
| Challenge: | Existing dense retrieval methods have achieved notable progress, but their effectiveness in legal case retrieval remains limited. |
| Approach: | They propose a self-evolving framework for rule-driven query rewriting that enhances BM25 without any parameter training. |
| Outcome: | The proposed framework outperforms non-evolutionary baselines, including human-designed rules and greedy rule selection, especially when powered by a high-capacity core LLM. |
Copied to clipboard
| Challenge: | Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work. |
| Approach: | They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations. |
| Outcome: | The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work. |
Copied to clipboard
| Challenge: | Existing vision-language models are based on exactmatch based accuracy and its derivations to evaluate performance. |
| Approach: | They propose a toolkit that supports systematic benchmarking, analysis, and interpretation of vision-language models by extracting intermediate outputs from any layer during the forward pass of open-source VLMs. |
| Outcome: | The proposed toolkit supports 16 state-of-the-art base VLMs and their over 30 variants and is extensible to accommodate new models without changing the core logic. |
Copied to clipboard
| Challenge: | Out-of-Domain (OOD) Intent Classification and New Intent Discovering are two tasks in the Task-Oriented Dialogue System. |
| Approach: | They propose a task paradigm to extend Out-of-Domain (OOD) Intent Classification and New Intent Discovering tasks in the Task-Oriented Dialogue System. |
| Outcome: | The proposed scheme improves on existing OOD intent classification and discovery datasets. |
Copied to clipboard
| Challenge: | *visual text representations* are a practical and surprisingly effective form of input compression for decoder LLMs. |
| Approach: | They exploit visual representations to render long text inputs as a single image and provide it directly to the model. |
| Outcome: | The proposed method reduces token usage while preserving performance. |
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have achieved significant performance in complex reasoning tasks such as mathematics and code generation. |
| Approach: | They propose a process-level benchmark specifically designed to assess the fine-grained error detection capabilities of PRMs. |
| Outcome: | The proposed model measures the accuracy, soundness, and sensitivity of 25 models across open-source and closed-source large language models. |
Copied to clipboard
| Challenge: | Abstract Meaning Representation parsing is a sentence-to-graph prediction task . graph nodes are semantically based on one or more sentence tokens, so implicit alignments can be derived. |
| Approach: | They propose a transition-based system that decouples hard-attention over sentences with a target-side action pointer mechanism to decouple source tokens from node representations and address alignments. |
| Outcome: | The proposed system achieves the second best Smatch score on AMR 2.0 (81.8) it decouples source tokens from node representations and addresses alignments, but lacks expressiveness. |
Copied to clipboard
| Challenge: | Current methods for information extraction (IE) focus on integrating IE output with the database . a long-overlooked question is what counts as "relevant knowledge" |
| Approach: | They propose a task that emphasizes integration of IE output and the database . they introduce a benchmark and an LLM agent framework for this task . |
| Outcome: | The proposed task integrates IE output and the target database (or knowledge base) it meets common demands such as data infilling, row population, and column addition . |
Copied to clipboard
| Challenge: | Recent work shows that pre-trained sequence-to-sequence Transformer models are effective in predicting linearized Abstract Meaning Representation graphs. |
| Approach: | They propose a structure-aware transition-based approach to AMR parsing that integrates general pre-trained sequence-to-sequence language models with a structured transition set. |
| Outcome: | The proposed approach retains the desirable properties of previous approaches while reaching the new parsing state of the art for AMR 2.0. |
Copied to clipboard
| Challenge: | Existing methods for medical visual question answering lack robustness and reasoning paths for real-world medical diagnostics. |
| Approach: | They propose a hierarchical expert verification reasoning chain method to enhance interpretability and accuracy in medical visual question answering. |
| Outcome: | The proposed method outperforms existing methods on four standard Med-VQA datasets. |
Copied to clipboard
| Challenge: | Neural machine translation is usually done via knowledge distillation from an autoregressive (AR) model. |
| Approach: | They leverage large monolingual corpora to improve the NAR model's performance by transferring the autoregressive model' s generalization ability while preventing overfitting. |
| Outcome: | The proposed methods on the WMT14 En-De and WMT16 En-Ro news translation tasks show that monolingual data augmentation improves the NAR model to approach the teacher AR model’s performance. |
Copied to clipboard
| Challenge: | Using data from 420k Twitter posts, we characterize anti-Asian violence-provoking speech and collect a community-crowdsourced dataset to facilitate its large-scale detection. |
| Approach: | They develop a codebook to characterize anti-Asian violence-provoking speech and collect a community-crowdsourced dataset to facilitate its large-scale detection. |
| Outcome: | The proposed codebook analyzes 420k tweets over 3 years and compares classifiers with hateful speech classifier classifier to detect hateful content. |
Copied to clipboard
| Challenge: | Existing approaches to long-term memory management rely on pairwise relations, causing fragmented retrieval. |
| Approach: | They propose a hypergraph-based hierarchical memory architecture that explicitly models high-order associations using hyperedges. |
| Outcome: | Experiments show that HyperMem achieves state-of-the-art performance with 92.73% accuracy for long-term conversations. |
Copied to clipboard
| Challenge: | Existing methods to extract triplets for unseen relations rely on costly fine-tuning and lack structured semantic guidance. |
| Approach: | They propose a framework that adopts a "frame first, then extract" paradigm to extract triplets from unstructured text. |
| Outcome: | The proposed framework achieves competitive zero-shot performance on multiple benchmarks and can be used to enhance existing extraction methods. |
Copied to clipboard
| Challenge: | Abstract Meaning Representation parsers rely on node-to-word alignments, but lack the complexity of the pipeline. |
| Approach: | They propose a neural aligner for abstract meaning representation that learns node-to-word alignments without relying on pipelines. |
| Outcome: | The proposed approach improves accuracy and generalization from AMR2.0 to AMR3.0 corpora. |
Copied to clipboard
| Challenge: | Existing methods to train dense passage retrieval have a large data gap between upstream and downstream relevance. |
| Approach: | They propose a method to pre-train the dense retriever with the text relevance induced by hyperlinks within Web documents. |
| Outcome: | The proposed method outperforms existing methods under different scenarios and in the open-domain question answering domain. |
Copied to clipboard
| Challenge: | Existing retrievers suffer from temporal-semantic misalignment and outdated-document interference . Existing frameworks suffer from both temporal validity and outdated factual versions . |
| Approach: | They propose a framework that mitigates temporal hallucinations by embedding heterogeneous temporal signals into the semantic space to ensure retrieval fidelity. |
| Outcome: | Experiments show that Re3 outperforms baselines by 9.7% in generation accuracy . the framework outperformed strongest baselines on challenging dynamic tasks . |