Papers by Kosuke Nishida
Improving Few-Shot Image Classification Using Machine- and User-Generated Natural Language Descriptions (2022.findings-naacl)
Copied to clipboard
| Challenge: | Humans can efficiently learn about new concepts from language descriptions, and we propose a new machine learning model, LIDE, which has a text decoder to generate the descriptions and a decoded text encoder to obtain the text representations of machine-generated descriptions. |
| Approach: | They propose a model with a text decoder to generate the descriptions and a corresponding text encoder to obtain the text representations of machine- or user-generated descriptions. |
| Outcome: | The proposed model outperforms baseline models with machine-generated descriptions and with high-quality user-generated models with high quality explanations. |
Multi-style Generative Reading Comprehension (P19-1)
Copied to clipboard
Kyosuke Nishida, Itsumi Saito, Kosuke Nishida, Kazutoshi Shinoda, Atsushi Otsuka, Hisako Asano, Junji Tomita
| Challenge: | Current studies on generative reading comprehension (RC) focus on extracting an answer span from textual evidence and natural language generation (NLG). |
| Approach: | They propose a multi-style abstractive summarization model for question answering called Masque. |
| Outcome: | The proposed model achieves state-of-the-art performance on the Q&A and Q& A + NLG tasks of MS MARCO and NarrativeQA. |
Initialization of Large Language Models via Reparameterization to Mitigate Loss Spikes (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train large language models that require a non-uniform model norm are not effective. |
| Approach: | They propose a technique that allows for uniformity of the norm of the model parameters . they propose 'weight scaling as reparameterization' to adjust the norm to the parameter . |
| Outcome: | The proposed technique outperforms existing methods and stabilizes training with the transformer decoders. |
Task-adaptive Pre-training of Language Models with Word Embedding Regularization (2021.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models acquire domain-independent knowledge through pre-training with massive textual resources. |
| Approach: | They propose a task-adaptive pre-training process that makes static embeddings close to the word embedds obtained in the target domain. |
| Outcome: | The proposed process improves on BioASQ and SQuAD when the pre-training corpora were not dominated by indomain data. |
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction (P19-1)
Copied to clipboard
Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Atsushi Otsuka, Itsumi Saito, Hisako Asano, Junji Tomita
| Challenge: | Question answering (QA) using textual sources for purposes such as reading comprehension has attracted much attention. |
| Approach: | They propose a Query Focused Extractor model for evidence extraction and multi-task learning with the QA model. |
| Outcome: | The proposed model achieves state-of-the-art evidence extraction score on hotpotQA and FEVER, which is a recognizing textual entailment task on a large textual database. |
Debiasing Reward Models via Causally Motivated Inference-Time Intervention (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches for mitigating spurious features in RMs focus on response length . Existing methods focus on RM activation, resulting in performance trade-offs . |
| Approach: | They propose a method that uses neurons to suppress spurious features in RMs at inference time. |
| Outcome: | The proposed method reduces sensitivity to spurious features without inducing performance trade-offs on RM benchmarks. |
Self-Adaptive Named Entity Recognition by Retrieving Unstructured Knowledge (2023.eacl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is costly because of lack of training data and domain experts. |
| Approach: | They propose a self-adaptive neural model that retrieves external knowledge from unstructured text to learn the usages of entities that have not been learned well. |
| Outcome: | The proposed model outperforms strong baselines on cross-neuro-ner datasets by 2.35 points in F1 metric. |
Unsupervised Domain Adaptation of Language Models for Reading Comprehension (2020.lrec-1)
Copied to clipboard
| Challenge: | State-of-the-art reading comprehension models do not have general linguistic intelligence . accuracy of out-domain datasets is affected by the distribution of data . |
| Approach: | They propose to use supervised RC training data in the source domain and unlabeled passages in the target domain to adapt models. |
| Outcome: | The proposed model outperforms the model without domain adaptation with five datasets in different domains. |