Papers by Gabriel Murray
Explicit Bayesian Inference to Uncover the Latent Themes of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive generative capabilities, yet their inner mechanisms remain largely opaque. |
| Approach: | They propose a variational autoencoder-based neural topic model to interpret LLMs generation process through an explicit Bayesian framework by inferring latent topic variables via variational inference. |
| Outcome: | The proposed model outperforms state-of-the-art topic models on intrinsic measures of coherence and diversity on multiple datasets and shows significant gains on classification and summarization tasks. |
Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In recent years, pre-trained language models have become the de facto instrument for the field of natural language processing (NLP). |
| Approach: | They propose a method that injects linguistic structures into pre-trained language models in the parameter-efficient fine-tuning setting. |
| Outcome: | The proposed approach outperforms state-of-the-art methods with a comparable number of parameters. |
Diversity-Aware Coherence Loss for Improving Neural Topic Models (2023.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that our method significantly improves the performance of neural topic models without requiring any pretraining or additional parameters. |
| Approach: | They propose a variational autoencoder framework that minimizes the posterior and prior divergence and a diversity-aware coherence loss that encourages the model to learn corpus-level coherency scores while maintaining high diversity between topics. |
| Outcome: | The proposed approach significantly improves the performance of neural topic models without pretraining or additional parameters. |
Human Guided Exploitation of Interpretable Attention Patterns in Summarization and Topic Segmentation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have investigated the multi-head self-attention mechanism of transformers. |
| Approach: | They propose to use a human-in-the-loop pipeline to discover task-specific attention patterns and inject them into transformer models to improve their accuracy. |
| Outcome: | The proposed methods improve the performance of transformer models by incorporating predefined patterns into their attention matrices. |
Discourse Analysis and Its Applications (P19-4)
Copied to clipboard
| Challenge: | Discourse processing is a suite of NLP tasks to uncover linguistic structures from texts at several levels, which can support many downstream applications. |
| Approach: | They present a set of tasks to uncover linguistic structures from texts at several levels, which can support many downstream applications. |
| Outcome: | The tutorial covers the basic concepts of discourse analysis and linguistic structures in monologue vs. conversation, synchronous v. asynchronous conversation, and key linguistic structure in discourse analysis. |
NLP for Conversations: Sentiment, Summarization, and Group Dynamics (C18-3)
Copied to clipboard
| Challenge: | a tutorial focuses on computational models for conversational structure, summarization and sentiment detection, and group dynamics. |
| Approach: | a tutorial will provide examples of specific NLP tasks for conversational structure, summarization and sentiment detection, and group dynamics. |
| Outcome: | The tutorial focuses on the three areas of conversational structure, summarization and sentiment detection, and group dynamics. |