Papers by Sosuke Kobayashi
Unsupervised Learning of Style-sensitive Word Vectors (P18-2)
Copied to clipboard
| Challenge: | Existing studies on what is said and how it is said focus on stylistic variations . lack of objective definitions is a major difficulty in studying style . |
| Approach: | They propose to extend the continuous bag of words embedding model to learn style-sensitive word vectors using a wider context window. |
| Outcome: | The proposed extensions contribute to the acquisition of style-sensitive word embeddings. |
Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition (2020.acl-main)
Copied to clipboard
Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Ryuto Konno, Kentaro Inui
| Challenge: | Recent neural networks can induce good span feature representations and achieve high performance in structured prediction tasks. |
| Approach: | They propose an instance-based learning method that learns similarities between spans . they aim to build models that have high interpretability without sacrificing performance . |
| Outcome: | The proposed method improves interpretability without sacrificing performance. |
B2T Connection: Serving Stability and Performance in Deep Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to prevent the vanishing gradient problem in deep neural networks are not effective. |
| Approach: | They propose a method that can equip both higher stability and effective training by a simple modification from Post-LN. |
| Outcome: | The proposed method outperforms Pre-LN and Post-Ln on a wide range of tasks. |
Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations (N18-2)
Copied to clipboard
| Challenge: | Neural network-based models for NLP have been growing with state-of-the-art results in various tasks. |
| Approach: | They propose a data augmentation method for labeled sentences called contextual augmentation. |
| Outcome: | The proposed method improves classifiers based on convolutional or recurrent neural networks. |
Instance-Based Neural Dependency Parsing (2021.tacl-1)
Copied to clipboard
Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Masashi Yoshikawa, Kentaro Inui
| Challenge: | Existing models that use instance-based inference for dependency parsing are difficult to understand for humans. |
| Approach: | They develop neural models that adopt an interpretable inference process for dependency parsing. |
| Outcome: | The proposed models achieve competitive accuracy with standard neural models and have plausibility of instance-based explanations. |
Pointwise HSIC: A Linear-Time Kernelized Co-occurrence Norm for Sparse Linguistic Expressions (D18-1)
Copied to clipboard
| Challenge: | Empirically, PHSIC is learned thousands of times faster than an RNN-based PMI while outperforming PMI in accuracy. |
| Approach: | They propose a new kernel-based co-occurrence measure that can be applied to sparse linguistic expressions with a very short learning time. |
| Outcome: | The proposed measure can be applied to sparse linguistic expressions with a very short learning time, and is called the pointwise HSIC. |
SHAPE: Shifted Absolute Position Embedding for Transformers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing position representations suffer from a lack of generalization to test data with unseen lengths or high computational cost. |
| Approach: | They propose to achieve shift invariance by randomly shifting absolute positions during training by a SHAPE algorithm that is empirically comparable to its counterpart. |
| Outcome: | The proposed method outperforms existing representations on sequence-to-sequence tasks due to extrapolation, i.e., the ability to generalize to sequences that are longer than those observed during training. |