Papers by Sho Takase
Evaluation Dataset for Zero Pronoun in Japanese to English Translation (2020.lrec-1)
Copied to clipboard
| Challenge: | In natural language, we often omit some words that are easily understandable from the context. |
| Approach: | They propose to use a dataset to evaluate whether translation models can resolve zero pronoun problems in Japanese to English translations. |
| Outcome: | The proposed model can resolve the zero pronoun problem in Japanese to English translations. |
Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing subword regularizations use multiple segmentations during training but only use one segmentation in inference. |
| Approach: | They propose an inference strategy that uses multiple subword segmentations to solve this discrepancy in the training process and inference. |
| Outcome: | The proposed strategy reduces the cost of training and improves the performance of models trained with subword regularization in low-resource machine translation tasks. |
Large Vocabulary Size Improves Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have investigated the properties of internal layers in large language models, but no studies have defined the vocabulary size. |
| Approach: | They propose a method to use a new vocabulary instead of the pre-defined one in a continual training scenario. |
| Outcome: | The proposed method outperforms the model with the pre-defined vocabulary in a continual training scenario. |
An Empirical Study of Building a Strong Baseline for Constituency Parsing (P18-2)
Copied to clipboard
| Challenge: | Sequence-to-sequence models have been used for natural language generation tasks such as machine translation and summarization. |
| Approach: | They propose to build a strong baseline based on general purpose sequence-to-sequence models for constituency parsing. |
| Outcome: | The proposed model outperforms existing models in natural language generation tasks without any explicit task-specific knowledge or architecture of constituent parsing. |
Optimizing Word Segmentation for Downstream Task (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to optimize tokenizations for downstream tasks are not suitable for traditional NLP. |
| Approach: | They propose a method to explore a tokenization appropriate for a downstream task . they train a model to assign a high probability to such appropriate tokenization based on the downstream task loss . |
| Outcome: | The proposed method improves sentiment analysis and textual entailment tasks . it is also integrated into state-of-the-art contextualized embeddings and reports a positive effect . |
Multi-Task Learning for Cross-Lingual Abstractive Summarization (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies use pseudo cross-lingual abstractive summarization data to train neural encoder-decoders. |
| Approach: | They propose a multi-task learning framework for cross-lingual abstractive summarization that attaches a special token to the beginning of the input sentence to indicate the target task. |
| Outcome: | The proposed model achieves better performance than the model trained with only pseudo cross-lingual abstractive summarization data. |
Rethinking Perturbations in Encoder-Decoders for Fast Training (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have proposed various regularization methods to avoid over-fitting. |
| Approach: | They propose to use scheduled sampling and adversarial perturbations to regularize neural models but they are not efficient enough for training time. |
| Outcome: | The proposed methods achieve comparable scores even though they are faster. |
Dynamic Structured Neural Topic Model with Self-Attention Mechanism (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent topic models that capture the time-series evolution of topics assume that topics evolve independently without interaction. |
| Approach: | They propose a dynamic structured neural topic model which captures topic dependencies while capturing their dependencies. |
| Outcome: | The proposed model outperforms a prior dynamic embedded topic model regarding perplexity and coherence while maintaining sufficient diversity across topics. |
Generating Natural Anagrams: Towards Language Generation Under Hard Combinatorial Constraints (D19-1)
Copied to clipboard
| Challenge: | Existing methods for creating anagrams do not pay much attention to the naturalness of the generated anagramms. |
| Approach: | They propose to combine depth-first search with modern neural language models to generate anagrams by permutation of characters in an input sentence or phrase. |
| Outcome: | The proposed method generates significantly more natural anagrams than baseline methods. |
Direct Output Connection for a High-Rank Language Model (D18-1)
Copied to clipboard
| Challenge: | Neural network language models have played a central role in recent natural language processing advances. |
| Approach: | They propose a recurrent neural network language model that combines probability distributions computed from a final RNN layer and also middle layers. |
| Outcome: | The proposed language model achieves the best score on the Penn Treebank and WikiText-2 benchmark datasets. |
B2T Connection: Serving Stability and Performance in Deep Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to prevent the vanishing gradient problem in deep neural networks are not effective. |
| Approach: | They propose a method that can equip both higher stability and effective training by a simple modification from Post-LN. |
| Outcome: | The proposed method outperforms Pre-LN and Post-Ln on a wide range of tasks. |
Improving Truthfulness of Headline Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on abstractive summarization report ROUGE scores, but are concerned about the truthfulness of generated summaries. |
| Approach: | They propose to remove untruthful instances from supervision data to improve headline generation . they build a binary classifier that predicts an entailment relation between an article and its headline . |
| Outcome: | The proposed model improves on two popular datasets. |
Positional Encoding to Control Output Sequence Length (N19-1)
Copied to clipboard
| Challenge: | Neural encoder-decoder models have been successful in natural language generation tasks, but they must be limited to a specified length for abstractive summarization. |
| Approach: | They propose a sinusoidal positional extension to preserve the length constraint so that a neural encoder-decoder model can generate a text of any length even if the target length is unseen in training data. |
| Outcome: | The proposed method can generate a text of any length even if the target length is unseen in training data and improves ROUGE scores. |
Word-level Perturbation Considering Word Length and Compositional Subwords (2022.findings-acl)
Copied to clipboard
| Challenge: | Word replacement considering length and compositional word replacement are effective word-level perturbations. |
| Approach: | They propose two simple modifications for word-level perturbation: Word Replacement considering Length and Compositional Word Replacement. |
| Outcome: | The proposed methods improve word-level perturbation and classification performance. |
ExtraPhrase: Efficient Data Augmentation for Abstractive Summarization (2022.naacl-srw)
Copied to clipboard
| Challenge: | Recent studies indicated that neural methods are governed by the scaling law for the amount of training data. |
| Approach: | They propose a low-cost strategy to augment training data for abstractive summarization tasks by extracting summarized text and paraphrasing it. |
| Outcome: | The proposed strategy outperforms back-translation and self-training and is more cost-efficient when training data is small. |
Interpretability for Language Learners Using Example-Based Grammatical Error Correction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing neural-based GEC models mainly aim at improving accuracy, but their interpretability has not been explored. |
| Approach: | They propose an example-based method that generates corrections using retrieved examples. |
| Outcome: | The proposed method improves interpretability and supports language learners. |
Joint Optimization of Tokenization and Downstream Model (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have reported that an appropriate tokenization depends on each downstream task. |
| Approach: | They propose a method to find an appropriate tokenization to a downstream task by optimizing a tokenizer and a model. |
| Outcome: | The proposed method improves on text classification and machine translation tasks. |