Papers by Aru Maekawa
Generative Replay Inspired by Hippocampal Memory Indexing for Continual Language Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Continual learning (CL) is a fundamental requirement for human-like general intelligence (Parisi et al., 2019). |
| Approach: | They propose to control sample generation using compressed features of previous training samples by using hippocampal memory indexing to enhance the generative replay. |
| Outcome: | The proposed method outperforms current generative replay methods and generates training samples from previous tasks. |
DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to extract word embeddings from training datasets are not efficient for training other models. |
| Approach: | They propose a method to distill a training dataset into a textual model by combining a small number of informative synthetic samples. |
| Outcome: | The proposed method outperforms existing methods on training datasets and language models. |
Dataset Distillation with Attention Labels for Fine-tuning BERT (2023.acl-short)
Copied to clipboard
| Challenge: | Specifically, we propose to introduce attention labels, which can efficiently distill the knowledge from the original dataset and transfer it to the transformer models via attention probabilities. |
| Approach: | They propose to introduce attention labels which can efficiently distill the knowledge from the original dataset and transfer it to the transformer models via attention probabilities. |
| Outcome: | The proposed methods perform impressively in four different NLP tasks and achieve 93.2% accuracy in AGNews, which is 98.5% of the original dataset even with only one sample per class and only one gradient step. |
Can we obtain significant success in RST discourse parsing by using Large Language Models? (2024.eacl-long)
Copied to clipboard
| Challenge: | Experimental results show that LLMs with tens of billion parameters can perform discourse parsing tasks. |
| Approach: | They employ Llama 2 and fine-tune it with QLoRA to achieve similar results . they show that LLMs with tens of billion parameters can perform a wide range of NLP tasks . |
| Outcome: | The proposed model performs better than existing models on three benchmark datasets. |
Live Football Commentary System Providing Background Information (2025.acl-demo)
Copied to clipboard
Yuichiro Mori, Chikara Tanaka, Aru Maekawa, Satoshi Kosugi, Tatsuya Ishigaki, Kotaro Funakoshi, Hiroya Takamura, Manabu Okumura
| Challenge: | Existing studies on sports commentary generation focus on describing major events in the video, but real-world commentary often includes background information. |
| Approach: | They developed an audio commentary system that generates utterances with background information and play-by-play commentary for football matches. |
| Outcome: | The proposed system generates utterances with background information and play-by-play commentary for football matches. |