Papers by Yingji Zhang
LangVAE and LangSpace: Building and Probing for Language Model VAEs (2025.emnlp-demos)
Copied to clipboard
| Challenge: | a novel framework for modular construction of variational autoencoders (VAEs) on pre-trained large language models (LLMs) is presented. |
| Approach: | They propose a modular framework for variational autoencoders on top of pre-trained large language models. |
| Outcome: | The proposed framework can encode pre-trained language models into more compact and semantically disentangled representations. |
Formal Semantic Controls over Language Models (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | Text embeddings provide a concise representation of the semantics of sentences and larger spans of text, rather than individual words, capturing a wide range of linguistic features. |
| Approach: | They propose to shorten the gap between latent semantics and formal symbolics by comparing distributional models to symbolic models grounded on formal linguistics and well-defined mathematical properties. |
| Outcome: | This paper examines the analysis and control of text representations, covering methods from pooling to LLM-based. |
Graph-Induced Syntactic-Semantic Spaces in Transformer-Based Variational AutoEncoders (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies on syntactic injection in Variational AutoEncoders (VAEs) are limited to LSTM-based VAEs. |
| Approach: | They propose to use latent space separation techniques to inject syntactic information into Variational AutoEncoders (VAEs) using graph-based models. |
| Outcome: | The proposed end-to-end VAE architecture can improve the organisation of the latent space, alleviating the information loss occurring in standard VAE setups, and resulting in enhanced performances on language modelling and downstream generation tasks. |
Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks (2024.acl-long)
Copied to clipboard
| Challenge: | Disentangled latent spaces usually have better semantic separability and geometrical properties, which leads to better interpretability and controllable data generation. |
| Approach: | They propose a flow-based invertible neural network mechanism integrated with a transformer-based language Autoencoder to deliver latent spaces with better semantic separability and geometrical properties. |
| Outcome: | The proposed model can deliver latent spaces with better separability properties compared to the current state-of-the-art models. |
Learning Disentangled Representations for Natural Language Definitions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. |
| Approach: | They propose to use syntactic and semantic regularities in textual data to provide models with both structural biases and generative factors. |
| Outcome: | The proposed model outperforms baselines on several qualitative and quantitative benchmarks and improves the results in the downstream task of definition modeling. |
Improving Semantic Control in Discrete Latent Spaces with Transformer Quantized Variational Autoencoders (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent work has struggled to achieve consistent results due to the inevitable loss of semantic information in the variational bottleneck and limited control over the decoding mechanism. |
| Approach: | They propose a model that leverages the controllability of VQVAE to guide the self-attention mechanism in Transformer-based VAEs to improve semantic control and generation. |
| Outcome: | The proposed model outperforms existing state-of-the-art VAE models in terms of control and preservation of semantic information across different tasks. |