Papers by Masaru Isonuma
Unsupervised Abstractive Opinion Summarization by Generating Sentences with Tree-Structured Topic Guidance (2021.tacl-1)
Copied to clipboard
| Challenge: | Abstractive summarization is a novel method for opinionated texts . it uses a recursive Gaussian mixture to generate topic sentences . |
| Approach: | They propose an unsupervised abstractive summarization method for opinionated texts . they alternate the unimodal Gaussian prior with a recursive Gausssian mixture . |
| Outcome: | The proposed method generates topic sentences with tree-structured topic guidance, which are more informative and cover more input contents than the current model. |
SciReviewGen: A Large-scale Dataset for Automatic Literature Review Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing literature review models have addressed literature review generation, but lack of large-scale datasets has been a stumbling block. |
| Approach: | They propose to use a large-scale dataset to evaluate automatic literature review generation models. |
| Outcome: | The proposed model can generate summaries comparable to human-written reviews while lacking detailed information. |
Unsupervised Neural Single-Document Summarization of Reviews via Learning Latent Discourse Structure and its Ranking (P19-1)
Copied to clipboard
| Challenge: | Currently, unsupervised summarization is widely used for product reviews on E-commerce websites. |
| Approach: | They propose an unsupervised model that learns the latent discourse tree without an external parser and generates a concise summary. |
| Outcome: | The proposed model outperforms other unsupervised approaches for relatively long reviews and is competitive with or better than supervised models. |
Unlearning Traces the Influential Training Data of Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit remarkable abilities without being explicitly trained for such tasks. |
| Approach: | They propose a method that unlearns a test dataset and evaluates the unlearned model on training datasets. |
| Outcome: | The proposed method resembles UnTrac, while being efficient for massive training datasets. |
Topic Modeling for Short Texts with Large Language Models (2024.acl-srw)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can be used to solve topic modeling challenges for short texts by contextually learning the meanings of words. |
| Approach: | They propose two approaches to using Large Language Models (LLMs) for topic modeling: parallel prompting and sequential prompting. |
| Outcome: | The proposed methods identify more coherent topics than existing ones while maintaining the diversity of the induced topics. |
Lexical Entailment with Hierarchy Representations by Deep Metric Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing lexical entailment studies cannot be applied to words that are not included in the training dataset. |
| Approach: | They propose a method that learns a mapping from word embeddings to hierarchical embedds to predict hypernymy relations among words. |
| Outcome: | The proposed method achieves state-of-the-art performance and robustness for unknown words. |
Dynamic Structured Neural Topic Model with Self-Attention Mechanism (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent topic models that capture the time-series evolution of topics assume that topics evolve independently without interaction. |
| Approach: | They propose a dynamic structured neural topic model which captures topic dependencies while capturing their dependencies. |
| Outcome: | The proposed model outperforms a prior dynamic embedded topic model regarding perplexity and coherence while maintaining sufficient diversity across topics. |
Tree-Structured Neural Topic Model (2020.acl-main)
Copied to clipboard
| Challenge: | Existing topic models do not organize topics into coherent groups or hierarchies. |
| Approach: | They propose a tree-structured neural topic model with an infinite number of branches and a topic distribution over a forest. |
| Outcome: | The proposed model improves data scalability and competitive performance when inducing latent topics and tree structures. |
How a Bilingual LM Becomes Bilingual: Tracing Internal Representations with Sparse Autoencoders (2025.findings-emnlp)
Copied to clipboard
Tatsuro Inaba, Go Kamoda, Kentaro Inui, Masaru Isonuma, Yusuke Miyao, Yohei Oseki, Yu Takagi, Benjamin Heinzerling
| Challenge: | Using sparse autoencoders, we explore how bilingual language models develop complex internal representations. |
| Approach: | They employ sparse autoencoders to analyze bilingual language models' internal representations. |
| Outcome: | The proposed method integrates decomposed representations from a fully trained model into a mid-training model. |
Differentiable Instruction Optimization for Cross-Task Generalization (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that instruction tuning is effective for generalizing to arbitrary tasks unseen during training. |
| Approach: | They propose to introduce learnable instructions and optimize them with gradient descent to optimize instruction for generalization ability. |
| Outcome: | The proposed instruction extractor extracts appropriate instruction and improves generalization ability compared to manual instruction tuning. |
Instability in Downstream Task Performance During LLM Pretraining (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a study of large language models shows that task scores fluctuate throughout training . |
| Approach: | They empirically analyze the stability of downstream task performance in an LLM . |
| Outcome: | The proposed methods improve performance stability without changes to the training procedure. |