Papers by Atsushi Saito
RWKV: Reinventing RNNs for the Transformer Era (2023.findings-emnlp)
Copied to clipboard
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan Wind, Stanisław Woźniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu
| Challenge: | recurrent neural networks struggle to match the performance of Transformers due to limitations in parallelization and scalability. |
| Approach: | They propose a model architecture that combines the efficient parallelizable training of transformers with the efficient inference of RNNs. |
| Outcome: | The proposed model performs on par with similarly sized RNNs, suggesting future work can leverage this architecture to create more efficient models. |
Multi-style Generative Reading Comprehension (P19-1)
Copied to clipboard
Kyosuke Nishida, Itsumi Saito, Kosuke Nishida, Kazutoshi Shinoda, Atsushi Otsuka, Hisako Asano, Junji Tomita
| Challenge: | Current studies on generative reading comprehension (RC) focus on extracting an answer span from textual evidence and natural language generation (NLG). |
| Approach: | They propose a multi-style abstractive summarization model for question answering called Masque. |
| Outcome: | The proposed model achieves state-of-the-art performance on the Q&A and Q& A + NLG tasks of MS MARCO and NarrativeQA. |
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction (P19-1)
Copied to clipboard
Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Atsushi Otsuka, Itsumi Saito, Hisako Asano, Junji Tomita
| Challenge: | Question answering (QA) using textual sources for purposes such as reading comprehension has attracted much attention. |
| Approach: | They propose a Query Focused Extractor model for evidence extraction and multi-task learning with the QA model. |
| Outcome: | The proposed model achieves state-of-the-art evidence extraction score on hotpotQA and FEVER, which is a recognizing textual entailment task on a large textual database. |