Papers by Takumi Ito
STEP: Staged Parameter-Efficient Pre-training for Large Language Models (2025.naacl-short)
Copied to clipboard
| Challenge: | Recent LLM development trends involve pre-training models with a vast number of parameters on massive datasets. |
| Approach: | They propose a method that integrates parameter-efficient tuning techniques with model growth to reduce memory requirements while maintaining equivalent performance. |
| Outcome: | The proposed method reduces memory requirements by 53.9% while maintaining equivalent performance to vanilla pre-trained models on downstream tasks. |
Langsmith: An Interactive Academic Text Revision System (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Currently, diversity and inclusion initiatives in the academic community are encouraged . however, writing papers in English can be a daunting task . |
| Approach: | They propose a system that helps non-native English speakers to write papers in English . the system can suggest fluent, academic-style sentences based on their rough, incomplete phrases or sentences . |
| Outcome: | The proposed system can help non-native English speakers write papers in English . the system can suggest fluent, academic-style sentences based on their rough sentences . |
STEP: Staged Parameter-Efficient Pre-training for Large Language Models (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for reducing computational costs during pre-training have been studied, but they often degrade performance under fair conditions. |
| Approach: | They propose a method that combines parameter-efficient tuning and staged training to reduce memory requirements while maintaining comparable performance. |
| Outcome: | The proposed method reduces memory requirements by 40.4% while maintaining comparable performance. |
Investigating the Effectiveness of Multiple Expert Models Collaboration (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Using multi-domain MT, we compare the performance of a single model with a multi-expert model in a fair condition. |
| Approach: | They propose to combine a multi-domain machine translation model with a aggregation strategy to investigate their results. |
| Outcome: | The proposed approach outperforms the current multi-domain model and aggregation methods in a fair condition on multiple domain datasets. |
Language Models as an Alternative Evaluator of Word Order Hypotheses: A Case Study in Japanese (2020.acl-main)
Copied to clipboard
| Challenge: | a method using neural language models (LMs) for analyzing the word order of language is currently lacking. |
| Approach: | They propose a method using neural language models to analyze the word order in Japanese . they test whether there is a parallel between LMs and human word order preference . |
| Outcome: | The proposed method is validated by comparing it with other linguistic studies. |
Lower Perplexity is Not Always Human-Like (2021.acl-long)
Copied to clipboard
| Challenge: | Existing efforts to build human-like computational models have focused on English . a cross-lingual evaluation is needed to build such models, but current research has focused on Japanese . |
| Approach: | They re-examine an established generalization that lower perplexity is not always human-like in Japanese . they propose a cross-lingual evaluation to build human-type computational models . |
| Outcome: | The proposed model lacks universality and lower perplexity is not always human-like . the results suggest a cross-lingual evaluation will be necessary to build human-type models . |
TEASPN: Framework and Protocol for Integrated Writing Assistance Environments (D19-3)
Copied to clipboard
| Challenge: | TEASPN is an open-source protocol for integrated writing assistance environments . authors propose that developers and researchers can integrate the latest developments in natural language processing with low cost. |
| Approach: | They propose a protocol and framework for integrating writing aids with writing software. |
| Outcome: | The proposed protocol standardizes the way writing software communicates with servers that implement such technologies, allowing developers and researchers to integrate the latest developments in natural language processing (NLP) with low cost. |
On Entity Identification in Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing work has shed light on the internal mechanisms of language models that can recall factual knowledge composed of entities and relations. |
| Approach: | They propose a framework analogous to clustering quality metrics to analyze the correspondence between entities and their mentions. |
| Outcome: | The proposed framework is analogous to clustering quality metrics. |