Papers by Zining Zhu

Conceptual Hierarchies within LLMs (2026.findings-acl)

Copied to clipboard

Challenge:	Existing literature has explored abstraction within large language models (LLMs).
Approach:	They generate a dataset of semantic hierarchies and investigate their storage locations in six LLMs using activation patching, a causal intervention technique.
Outcome:	The results show that concepts at finer levels of granularity are stored around 61-78% of the time before those at coarser levels.

The Illusion of Specialization: Unveiling the Domain-Invariant "Standing Committee" in Mixture-of-Experts Models (2026.acl-long)

Copied to clipboard

Yan Wang, Yitao Xu, Nanhan Shen, Jinyan Su, Jimin Huang, Zining Zhu

Challenge:	Mixture of Experts models are widely assumed to achieve domain specialization through sparse routing.
Approach:	They propose a framework that analyzes routing behavior at the level of expert groups rather than individual experts.
Outcome:	The proposed framework analyzes routing behavior at the level of expert groups rather than individual experts.

ACCORD: Closing the Commonsense Measurability Gap (2025.naacl-long)

Copied to clipboard

François Roewer-Després, Jinyue Feng, Zining Zhu, Frank Rudzicz

Reasoning

Challenge:	ACCORD is a framework and benchmark suite for disentangling the commonsense grounding and reasoning abilities of large language models (LLMs).
Approach:	They propose a framework and benchmark suite for disentangling the commonsense grounding and reasoning abilities of large language models (LLMs) they introduce formal elements to explicitly control and quantify reasoning complexity beyond the typical 1 or 2 hops.
Outcome:	The proposed framework can scale with future LLM improvements.

How is BERT surprised? Layerwise detection of linguistic anomalies (2021.acl-long)

Copied to clipboard

Bai Li, Zining Zhu, Guillaume Thomas, Yang Xu, Frank Rudzicz

Semantic Text Processing

Challenge:	a number of studies have shown that transformer-based language models detect when a word is anomalous in context, but likelihood scores do not tell the cause of the anomaly.
Approach:	They propose to use Gaussian models for density estimation at intermediate layers of three language models to evaluate grammaticality.
Outcome:	The proposed method on BLiMP shows that language models employ different mechanisms to detect different types of linguistic anomalies.

Predicting Fine-Tuning Performance with Probing (2022.emnlp-main)

Copied to clipboard

Zining Zhu, Soroosh Shahtalebi, Frank Rudzicz

Language Models

Challenge:	Large-scale neural models have recently demonstrated impressive performance in language understanding tasks, typically evaluated by their fine-tuned performance.
Approach:	They propose to use probing to extract a proxy signal widely used in model development to predict fine-tuning performance.
Outcome:	The proposed method predicts fine-tuning performance with errors 40% - 80% smaller than baselines.

An unsupervised framework for tracing textual sources of moral change (2021.findings-emnlp)

Copied to clipboard

Aida Ramezani, Zining Zhu, Frank Rudzicz, Yang Xu

Responsible & Trustworthy NLP

Challenge:	Existing studies on moral sentiment classification and temporal inference of moral sentiment have not quantified the origins of these changes.
Approach:	They propose an unsupervised framework for tracing textual sources of moral change toward entities through time.
Outcome:	The proposed framework captures fine-grained human moral judgments and identifies coherent source topics of moral change triggered by historical events.

Neural reality of argument structure constructions (2022.acl-long)

Copied to clipboard

Bai Li, Zining Zhu, Guillaume Thomas, Frank Rudzicz, Yang Xu

Linguistics & Cognitive NLP

Challenge:	lexicalist linguistic theories assume argument structure is predictable from meaning of verbs . construction grammarians propose argument structure constructions distinct from verbs.
Approach:	They adapt psycholinguistic studies to probe for the existence of argument structure constructions in Transformer-based language models.
Outcome:	The proposed method could be used to probe argument structure constructions in LMs . the study shows that LM learners prefer grouping by construction over verb grouping .

INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent (2025.acl-long)

Copied to clipboard

Haohang Li, Yupeng Cao, Yangyang Yu, Shashidhar Reddy Javaji, Zhiyang Deng, Yueru He, Yuechen Jiang, Zining Zhu, K.p. Subbalakshmi, Jimin Huang, Lingfei Qian, Xueqing Peng, Jordan W. Suchow, Qianqian Xie

Semantic Text Processing

Challenge:	Recent advances have underscored the potential of large language model (LLM)-based agents in financial decision-making.
Approach:	They propose to evaluate LLM agents using 13 different LLMs as backbone models across various market environments and tasks.
Outcome:	The proposed framework assesses the reasoning and decision-making capabilities of 13 different LLMs across various market environments and tasks.

On the data requirements of probing (2022.findings-acl)

Copied to clipboard

Zining Zhu, Jixuan Wang, Bai Li, Frank Rudzicz

Challenge:	Existing methods to probe neural networks are expensive and require large datasets.
Approach:	They propose a method to estimate the required number of data samples in probing datasets . they use a classification task to encode a text with a deep neural network .
Outcome:	The proposed method estimates the required number of data samples in two probing configurations and proves it is statistically powerful.

Explanation in the Era of Large Language Models (2024.naacl-tutorials)

Copied to clipboard

Zining Zhu, Hanjie Chen, Xi Ye, Qing Lyu, Chenhao Tan, Ana Marasovic, Sarah Wiegreffe

Explainability & Interpretability in NLP

Challenge:	Explanation has long been a part of communication, where humans use language to elucidate each other and transmit information about mechanisms of events.
Approach:	They review the opportunities and challenges of explanations in the era of large language models and examine how they can be used to generate explanations.
Outcome:	The proposed methods are based on the models of large language models (LLMs) and their opaque nature.

A State-Vector Framework for Dataset Effects (2023.emnlp-main)

Copied to clipboard

Esmat Sahak, Zining Zhu, Frank Rudzicz

Representation Learning

Challenge:	Recent DNN-based systems gain linguistic abilities on multiple levels ranging from syntax, semantics, and even some discourse-related abilities.
Approach:	They propose a state-vector framework that uses idealized probing test results as the bases of a vector space to quantify the effects of both standalone and interacting datasets.
Outcome:	The proposed framework allows to quantify the effects of both standalone and interacting datasets.

An information theoretic view on selecting linguistic probes (2020.emnlp-main)

Copied to clipboard

Zining Zhu, Frank Rudzicz

Text Classification

Challenge:	Recent advances in NLP tasks require a question of how much linguistic knowledge is encoded in neural networks.
Approach:	They propose to use diagnostic classifiers to perform supervised classification from internal representations.
Outcome:	Empirically, the two proposed criteria lead to results that agree with each other.

Distribution Prompting: Understanding the Expressivity of Language Models Through the Next-Token Distributions They Can Produce (2025.emnlp-main)

Copied to clipboard

Haojin Wang, Zining Zhu, Freda Shi

Language Models

Challenge:	Autoregressive neural language models (LMs) generate a probability distribution over tokens at each time step given a prompt.
Approach:	They propose to find a prompt that induces LMs to output a distribution as close as possible to the target, using either soft or hard gradient-based prompt tuning.
Outcome:	The proposed model is able to generate a distribution as close as possible to a target given a prompt, and it can be used to approximate distributions with low or high entropy.

Detecting cognitive impairments by agreeing on interpretations of linguistic features (N19-1)

Copied to clipboard

Zining Zhu, Jekaterina Novikova, Frank Rudzicz

Explainability & Interpretability in NLP

Challenge:	Linguistic features have shown promising applications for detecting cognitive impairments.
Approach:	They propose a framework to classify after reaching agreements between modalities by using linguistic features to divide linguistic subsets into subset and let neural networks learn low-dimensional representations that agree with each other.
Outcome:	The proposed framework outperforms existing classifiers using all of the 413 linguistic features.

Sheaf Discovery with Joint Computation Graph Pruning and Flexible Granularity (2025.emnlp-main)

Copied to clipboard

Lei Yu, Jingcheng Niu, Zining Zhu, Xi Chen, Gerald Penn

Multimodality

Challenge:	Experimental results show that DiscoGP extracts sheaves that preserve 93-100% of a model’s performance while comprising only 1-7% of the original weights and connections.
Approach:	They propose a framework for extracting self-contained modular units within neural language models (LMs) they use a gradient-based pruning algorithm to prune the original LM to a sparse skeleton .
Outcome:	The proposed framework preserves 93-100% of the original model's performance while preserving only 1-7% of the model''s original weights and connections.