Papers by Atsuki Yamaguchi
Enhancing Linguistic Competence of Language Models through Pre-training with Language Learning Tasks (2026.acl-short)
Copied to clipboard
| Challenge: | Language models (LMs) are pre-trained on raw text datasets to generate text sequences token-by-token. |
| Approach: | They propose a framework that integrates Language Learning Tasks alongside standard next-token prediction to stimulate the acquisition of morphological, syntactic, and semantic knowledge. |
| Outcome: | The proposed framework improves performance on linguistic competence benchmarks while maintaining competitive performance on reasoning tasks. |
Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updates (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models underperform in languages absent or underrepresented in training data, creating barrier to equitable access for speakers worldwide. |
| Approach: | They propose a selective parameter update strategy that proactively preserves source knowledge by identifying critical parameters critical to maintaining source abilities. |
| Outcome: | Experiments in five typologically diverse languages show that SSU mitigates catastrophic forgetting. |
EACL 2026 Student Research Workshop: Mentorship Program Report (2026.eacl-srw)
Copied to clipboard
| Challenge: | The SRW Mentorship Program provides constructive, formative guidance to student authors, especially first-time submitters, by pairing them with experienced researchers. |
| Approach: | This report provides a summary and analysis of the EACL 2026 Student Research Workshop (SRW) Mentorship Program, using structured exit surveys collected from mentors and mentees. |
| Outcome: | The findings will help clarify the organization of mentorship at *ACL venues and provide empirical data for future chairs. |
How do different tokenizers perform on downstream tasks in scriptio continua languages?: A case study in Japanese (2023.acl-srw)
Copied to clipboard
| Challenge: | Existing studies on scriptio continua languages lack comprehensiveness of tokenizers . authors use Byte-Pair-Encoding or Unigram instead of WordPiece for subword tokenizer . |
| Approach: | They investigate the effect of tokenizers on the downstream performance of pretrained language models in scriptio continua languages where no explicit spaces exist between words. |
| Outcome: | The proposed tokenizers perform better on a wide range of tasks compared with other tokenizer methods . the results show that each task has an optimal morphological analyzer . |
How does the task complexity of masked pretraining objectives affect downstream performance? (2023.findings-acl)
Copied to clipboard
| Challenge: | Masked language modeling (MLM) is a widely used self-supervised pretraining objective. |
| Approach: | They propose to use a mask-based objective to predict a token that is replaced with a masked token given its context. |
| Outcome: | The proposed objectives show that they should have half the complexity needed to perform comparably to MLM. |
JFLD: A Japanese Benchmark for Deductive Reasoning Based on Formal Logic (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have proficiently solved a broad range of tasks with their rich knowledge but struggle with logical reasoning. |
| Approach: | They propose a deductive reasoning benchmark for Japanese that assesses logical reasoning abilities isolated from knowledge and various reasoning rules. |
| Outcome: | The proposed benchmarks assess whether LLMs can generate logical steps to (dis)prove a given hypothesis based on a set of facts. |
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Cross-lingual vocabulary adaptation (CVA) methods have been proposed for adapting models to a target language . but effectiveness of these methods on increasing inference efficiency of generative large language models has not been explored. |
| Approach: | They propose to use cross-lingual vocabulary adaptation methods to adapt models to a target language to improve downstream performance. |
| Outcome: | The proposed methods significantly speed up models in four languages and four natural language understanding tasks. |
Rethinking the Idiomaticity Decomposability Hypothesis: Evidence from Distributional Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Decomposability is thought to predict syntactic flexibility, but is not attributed to distributional experience. |
| Approach: | They propose a model-internal measure of decomposability and relate it to human ratings, syntactic flexibility, and predictability while tracking idiom learning during pretraining. |
| Outcome: | The proposed model-internal measure correlates weakly with human judgments and shows a small but consistent negative relationship with syntactic flexibility. |
Dialogue Act-based Breakdown Detection in Negotiation Dialogues (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have succeeded in modeling a negotiating agent in natural language that can control both text generation and reasoning in goal-oriented dialogue systems. |
| Approach: | They propose a human-human negotiation dialogue dataset that features increased complexities in terms of the number of possible solutions and a utility function. |
| Outcome: | The proposed method performs comparable to text-based approaches in existing corpora and better results in the proposed dataset. |
Frustratingly Simple Pretraining Alternatives to Masked Language Modeling (2021.emnlp-main)
Copied to clipboard
| Challenge: | Masked language modeling (MLM) is widely used in natural language processing for self-supervised learning of text representations. |
| Approach: | They propose to use token-level classification tasks as main pretraining objectives instead of Masked language modeling (MLM) . Empirical results show that pretraining a model with 41% of the BERT-BASE’s parameters, BERT MEDIUM results in only a 1% drop in GLUE scores with their best objective. |
| Outcome: | Empirical results show that the proposed methods achieve comparable or better performance to MLM using a BERT-BASE architecture. |