A User-Centered, Interactive, Human-in-the-Loop Topic Modelling System (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent research has demonstrated the value of user feedback, but there are still issues to consider, such as the difficulty in tracking changes and comparing different models. |
| Approach: | They propose a human-in-the-loop topic modeling system that integrates users’ knowledge into the modelling process, enabling them to refine the model iteratively. |
| Outcome: | The proposed system is based on a series of user studies to assess its performance in progressively more realistic applications. |
Similar Papers
Why Didn’t You Listen to Me? Comparing User Control of Human-in-the-Loop Topic Models (P19-1)
Copied to clipboard
| Challenge: | Informed prior-based methods provide better control than constraints, but constraints yield higher quality topics, but with less control. |
| Approach: | They propose to use constraints and informed prior-based methods to improve user control and topic coherence. |
| Outcome: | The proposed methods improve user control and topic coherence, while constraints yield higher quality topics, but with less control. |
ITMT: Interactive Topic Model Trainer (2023.eacl-demo)
Copied to clipboard
| Challenge: | Topic Modeling is a commonly used technique for analyzing unstructured data, but achieving accurate results and useful models can be challenging. |
| Approach: | They propose to use an interactive topic model trainer to train and curation topic extraction libraries and compare it with other tools for topic modeling analysis. |
| Outcome: | The proposed tool is compared with other tools for topic modeling analysis. |
TopicGPT: A Prompt-based Topic Modeling Framework (2024.naacl-long)
Copied to clipboard
| Challenge: | TopicGPT uses large language models to uncover latent topics in text . topic models represent topics as bags of words that require "reading the tea leaves" topic models also offer limited control over formatting and specificity of topics . |
| Approach: | TopicGPT uses large language models to uncover latent topics in text . authors propose a prompt-based framework that produces topics that align better with human categorizations . |
| Outcome: | TopicGPT produces topics that align better with human categorizations compared to competing methods. |
A Query-Driven Topic Model (2021.findings-acl)
Copied to clipboard
| Challenge: | Topic modeling is an unsupervised method for revealing the hidden semantic structure of a corpus. |
| Approach: | They propose a query-driven topic model that allows users to specify a simple query in words or phrases and return query-related topics. |
| Outcome: | The proposed model is particularly attractive when the query has a low occurrence in a text corpus, making it difficult for traditional topic models to identify relevant topics. |
Are Neural Topic Models Broken? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation paradigms are often divorced from real-world use . recent results have challenged the validity of the prevailing model evaluation paradigm . |
| Approach: | They show that neural topic models fare worse in both respects compared to an established classical method. |
| Outcome: | The proposed method outperforms the members of the ensemble in both respects. |
Continually Improving Extractive QA via Human Feedback (2023.emnlp-main)
Copied to clipboard
| Challenge: | a study of extractive question answering systems using human feedback shows promising potential for continual learning. |
| Approach: | They study extractive question answering system by using user feedback to improve it . they design and deploy an iterative approach where users ask questions and provide feedback . |
| Outcome: | The proposed model improves over time across different data regimes and domains . human user feedback is more affordable and abundant than annotations provided by trained experts . |
Neural Topic Modeling with Large Language Models in the Loop (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated promising capabilities in topic discovery, but their direct application to topic modeling suffers from issues such as incomplete topic coverage, misalignment of topics, and inefficiency. |
| Approach: | They propose a novel LLM-in-the-loop framework that integrates Large Language Models with Neural Topic Models (NTMs) global topics and document representations are learned through the NTM, while an LLM refines these topics using an Optimal Transport (OT)-based alignment objective. |
| Outcome: | The proposed framework improves topic interpretability while preserving the efficiency of existing NTMs. |
Designing, Evaluating, and Learning from Humans Interacting with NLP Models (2023.emnlp-tutorial)
Copied to clipboard
| Challenge: | This tutorial will cover how to conduct human-in-the-loop usability evaluations to ensure that models are capable of interacting with humans. |
| Approach: | They will provide a systematic overview of key considerations and effective approaches for studying human-NLP model interactions. |
| Outcome: | This tutorial will cover how to conduct human-in-the-loop usability evaluations to ensure that models are capable of interacting with humans. |
Improving the TENOR of Labeling: Re-evaluating Topic Models for Content Analysis (2024.eacl-long)
Copied to clipboard
Zongxia Li, Andrew Mao, Daniel Stephens, Pranav Goel, Emily Walpole, Alden Dima, Juan Fung, Jordan Boyd-Graber
| Challenge: | Existing evaluation metrics such as coherence and coherency are inadequate for neural topic models. |
| Approach: | They conduct the first evaluation of neural, supervised and classical topic models in an interactive task-based setting. |
| Outcome: | The proposed model performs better on cluster evaluation metrics and human evaluations than classical models on real-world tasks. |
Topic Modeling: Contextual Token Embeddings Are All You Need (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current neural approaches to topic modeling have not been able to solve all of the problems. |
| Approach: | They propose a topic modeling approach that uses document contextual token embeddings to find topics and find topic spans within documents. |
| Outcome: | The proposed model outperforms the current state-of-the-art models on a comprehensive set of topic model evaluation metrics. |