Papers by Naoaki Okazaki

Controlled Generation with Prompt Insertion for Natural Language Explanations in Grammatical Error Correction (2024.lrec-main)

Challenge:	Existing work on news-image captioning requires a joint understanding of image and text.
Approach:	They propose a Transformer model that integrates text and image modalities and attends to textual features from visual features in generating a caption.
Outcome:	The proposed model outperforms the state-of-the-art model and improves the quality of news-image captions.

Copied to clipboard

Masahiro Kaneko, Naoaki Okazaki

Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation (2022.findings-acl)

Challenge:	Existing studies present tokens, examples, and hints for corrections, but do not directly explain the reasons in natural language.
Approach:	They propose a method called controlled generation with Prompt Insertion that uses Large Language Models to explain the reasons for corrections in natural language.
Outcome:	The proposed method can explain the reasons for corrections in natural language by guiding the LLMs to generate explanations for all correction points.

Copied to clipboard

Sho Takase, Tatsuya Hiraoka, Naoaki Okazaki

Machine Translation

Challenge:	Existing subword regularizations use multiple segmentations during training but only use one segmentation in inference.
Approach:	They propose an inference strategy that uses multiple subword segmentations to solve this discrepancy in the training process and inference.
Outcome:	The proposed strategy reduces the cost of training and improves the performance of models trained with subword regularization in low-resource machine translation tasks.

PatchBERT: Just-in-Time, Out-of-Vocabulary Patching (2020.emnlp-main)

Copied to clipboard

Sangwhan Moon, Naoaki Okazaki

Incorporating Semantic Attention in Video Description Generation (L18-1)

Challenge:	a pre-trained language model with low OOV can improve performance for transfer learning . a vocabulary surrogate can provide performance boosts with no additional computation cost .
Approach:	They propose multiple methods to mitigate OOV during downstream task fine-tuning . they demonstrate that vocabulary surrogates can provide performance boosts with no additional computation cost .
Outcome:	The proposed methods improve performance with the same parameter count when combined with fine-tuning.

Copied to clipboard

Natsuda Laokulrat, Naoaki Okazaki, Hideki Nakayama

Learning to Select, Track, and Generate for Data-to-Text (P19-1)

Challenge:	Existing methods to generate video descriptions fail to mention objects and actions in videos .
Approach:	They propose an LSTM-based sequence-to-sequence model with semantic attention mechanism for video description generation that includes external fine-grained visual information.
Outcome:	The proposed model can selectively focus on external fine-grained visual information and have a better quality of video descriptions.

Copied to clipboard

Hayate Iso, Yui Uehara, Tatsuya Ishigaki, Hiroshi Noji, Eiji Aramaki, Ichiro Kobayashi, Yusuke Miyao, Naoaki Okazaki, Hiroya Takamura

Social Bias Evaluation for Large Language Models Requires Prompt Variations (2025.findings-emnlp)

Challenge:	Existing models often refer to the same data record multiple times.
Approach:	They propose a data-to-text generation model with two modules, one for tracking and the other for text generation.
Outcome:	The proposed model outperforms existing models even without writer information in all evaluation metrics and contributes to content planning and surface realization.

Copied to clipboard

Rem Hida, Masahiro Kaneko, Naoaki Okazaki

Program Chairs’ Report on Peer Review at ACL 2023 (2023.acl-long)

Challenge:	Recent studies have tried to evaluate and mitigate social biases accurately using limited prompts.
Approach:	They investigate the sensitivity of Large Language Models when changing prompt variations . they found that LLM rankings fluctuate across prompts for both task performance and social bias .
Outcome:	The results show that LLM rankings fluctuate when changing prompt variations .

Copied to clipboard

Anna Rogers, Marzena Karpinska, Jordan Boyd-Graber, Naoaki Okazaki

Predicting Stances from Social Media Posts using Factorization Machines (C18-1)

Challenge:	ACL'23 makes its peer review report public and an official part of the conference proceedings.
Approach:	They present an analysis of the factors affecting peer review and identify the most problematic issues that the authors complained about.
Outcome:	The authors identified the most problematic issues and provided suggestions for the future chairs.

Copied to clipboard

Akira Sasaki, Kazuaki Hanawa, Naoaki Okazaki, Kentaro Inui

Challenge:	Social media provide platforms to express, discuss, and shape opinions about events and issues in the real world.
Approach:	They propose to use factorization machines to model user preferences toward topics from social media data to predict whether a given text/user is in favor (agree), against (disagreer), or neutral toward a target topic.
Outcome:	The proposed method can predict stances of silent users based on their stance toward other topics and the social media posts of the user.

Selective-LAMA: Selective Prediction for Confidence-Aware Evaluation of Language Models (2023.findings-eacl)

Copied to clipboard

Hiyori Yoshikawa, Naoaki Okazaki

Building a Japanese Document-Level Relation Extraction Dataset Assisted by Cross-Lingual Transfer (2024.lrec-main)

Challenge:	Recent studies suggest that neural language models learn and store a large amount of facts and commonsense knowledge from training data.
Approach:	They propose a benchmark task that evaluates the amount of relational knowledge stored in pre-trained language models.
Outcome:	The proposed evaluations show that the selection of confidence functions is more robust to simple guesses than the accuracy-based evaluation.

Copied to clipboard

Youmi Ma, An Wang, Naoaki Okazaki

Information Extraction & Text Mining

Challenge:	Document-level Relation Extraction (DocRE) is the task of extracting all semantic relationships from a document.
Approach:	They propose to transfer an English document to Japanese to promote DocRE in other languages.
Outcome:	The proposed model reduces the human edit steps by 50% compared with the previous approach.

Optimizing Word Segmentation for Downstream Task (2020.findings-emnlp)

Copied to clipboard

Tatsuya Hiraoka, Sho Takase, Kei Uchiumi, Atsushi Keyaki, Naoaki Okazaki

IMPARA: Impact-Based Metric for GEC Using Parallel Data (2022.coling-1)

Challenge:	Existing methods to optimize tokenizations for downstream tasks are not suitable for traditional NLP.
Approach:	They propose a method to explore a tokenization appropriate for a downstream task . they train a model to assign a high probability to such appropriate tokenization based on the downstream task loss .
Outcome:	The proposed method improves sentiment analysis and textual entailment tasks . it is also integrated into state-of-the-art contextualized embeddings and reports a positive effect .

Copied to clipboard

Koki Maeda, Masahiro Kaneko, Naoaki Okazaki

Sampling-based Pseudo-Likelihood for Membership Inference Attacks (2025.findings-acl)

Challenge:	Existing methods for automatic evaluation of grammatical error correction require multiple reference sentences or manual scores.
Approach:	They propose an Impact-based Metric for GEC using PARAllel data, IMPARA . IMPRA computes correction impacts computed by parallel data comprising pairs of grammatical/ungrammatically-spaced sentences.
Outcome:	The proposed method can perform evaluations that fit different domains and correction styles.

Copied to clipboard

Masahiro Kaneko, Youmi Ma, Yuki Wata, Naoaki Okazaki

Robustness in NLP

Challenge:	Large Language Models (LLMs) are trained on large-scale web data, which makes it difficult to grasp the contribution of each text.
Approach:	They propose a membership-inference attack method that uses only the input text to detect leaks.
Outcome:	The proposed method performs on par with existing likelihood-based methods even without likelihoods.

Gender Bias in Masked Language Models for Multiple Languages (2022.naacl-main)

Copied to clipboard

Masahiro Kaneko, Aizhan Imankulova, Danushka Bollegala, Naoaki Okazaki

Likelihood-based Mitigation of Evaluation Bias in Large Language Models (2024.findings-acl)

Challenge:	Masked Language Models (MLMs) pre-trained by predicting masked tokens on large corpora have been used successfully in natural language processing tasks for a variety of languages.
Approach:	They propose to use English attribute word lists to evaluate bias in eight languages without manually annotating data.
Outcome:	The proposed model significantly correlates with the existing English datasets for gender bias.

Copied to clipboard

Masanari Ohi, Masahiro Kaneko, Ryuto Koike, Mengsay Loem, Naoaki Okazaki

Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs (2021.tacl-1)

Challenge:	Large Language Models (LLMs) are widely used to evaluate natural language generation tasks as automated metrics.
Approach:	They propose to use LLMs to evaluate sentences with higher likelihoods and lower likelihoods to mitigate the likelihood bias.
Outcome:	The proposed method overrates sentences with higher likelihoods while underrating sentences with lower likelihoods.

Copied to clipboard

Emanuele Bugliarello, Ryan Cotterell, Naoaki Okazaki, Desmond Elliott

PLOG: Table-to-Logic Pretraining for Logical Table-to-Text Generation (2022.emnlp-main)

Challenge:	Large-scale pretraining and task-specific fine-tuning are now the standard methodology for many tasks in computer vision and natural language processing.
Approach:	They propose to combine two types of vision and language BERTs to create a theoretical framework that can be unified under different theoretical frameworks.
Outcome:	The proposed models can be classified into single-stream or dual-stream encoders and are unified under a single theoretical framework.

Copied to clipboard

Ao Liu, Haoyu Dong, Naoaki Okazaki, Shi Han, Dongmei Zhang

Semi-Supervised Formality Style Transfer with Consistency Training (2022.acl-long)

Challenge:	Logical table-to-text generation requires models to derive logical-level facts from table records via logical inference.
Approach:	They propose a pretrained logical form generator framework to improve generation fidelity . they use a dataset to test the logical inference accuracy of the framework .
Outcome:	The proposed framework outperforms baselines on LOGICNLG and CONTLOG on two benchmarks.

Copied to clipboard

Ao Liu, An Wang, Naoaki Okazaki

Low-Resource NLP

Challenge:	Formality style transfer (FST) is a task that involves paraphrasing an informal sentence into a formal one without altering its meaning.
Approach:	They propose a semi-supervised framework to better utilize source-side unlabeled sentences based on consistency training.
Outcome:	The proposed framework can achieve state-of-the-art results even with less than 40% of the parallel data.

Developing Japanese CLIP Models Leveraging an Open-weight LLM for Large-scale Dataset Translation (2025.naacl-srw)

Copied to clipboard

Issa Sugiura, Shuhei Kurita, Yusuke Oda, Daisuke Kawahara, Naoaki Okazaki

Multilinguality

Challenge:	lack of large-scale open Japanese image-text pairs poses a significant barrier to the development of vision-language models.
Approach:	They construct large-scale Japanese image-text pairs using machine translation and pre-trained CLIP models on a Japanese dataset.
Outcome:	The results show that pre-trained models achieve competitive average scores on Japanese culture tasks compared to models of similar size.

Learning How to Translate North Korean through South Korean (2022.lrec-1)

Copied to clipboard

Hwichan Kim, Sangwhan Moon, Naoaki Okazaki, Mamoru Komachi

Machine Translation

Challenge:	Existing NLP systems cannot properly handle North Korean inputs, despite limited data . Several NLP researchers have been working on the Korean language .
Approach:	They propose to manually create evaluation data for automatic alignment and machine translation, and investigate automatic alignment methods suitable for North Korean.
Outcome:	The proposed model trained by North Korean bilingual data significantly boosts translation accuracy compared to existing South Korean models in zero-shot settings.

Unsupervised Domain Adaptation for Sparse Retrieval by Filling Vocabulary and Word Frequency Gaps (2022.aacl-main)

Copied to clipboard

Hiroki Iida, Naoaki Okazaki

Multi-Task Learning for Cross-Lingual Abstractive Summarization (2022.lrec-1)

Challenge:	IR models with a pretrained language model outperform lexical approaches like BM25 for vocabulary mismatch.
Approach:	They propose an unsupervised domain adaptation method by filling vocabulary gaps by expanding queries and documents through an MLM.
Outcome:	The proposed method outperforms the current state-of-the-art domain adaptation method on datasets with a large vocabulary gap from a source domain.

Copied to clipboard

Sho Takase, Naoaki Okazaki

Query-based Image Captioning from Multi-context 360cdegree Images (2023.findings-emnlp)

Challenge:	Existing studies use pseudo cross-lingual abstractive summarization data to train neural encoder-decoders.
Approach:	They propose a multi-task learning framework for cross-lingual abstractive summarization that attaches a special token to the beginning of the input sentence to indicate the target task.
Outcome:	The proposed model achieves better performance than the model trained with only pseudo cross-lingual abstractive summarization data.

Copied to clipboard

Koki Maeda, Shuhei Kurita, Taiki Miyanishi, Naoaki Okazaki

Parameter-Efficient Korean Character-Level Language Modeling (2023.eacl-main)

Challenge:	Existing image captioning datasets follow salient images with limited camera field of view, ignoring minor details.
Approach:	They propose a task where a query specifies the context to describe in 360-degree images and construct a dataset for the task that contains 3,940 360- degree images and 18,459 pairs of queries and captions annotated manually.
Outcome:	The proposed task is more challenging than the conventional image captioning task, which describes salient objects in images.

Copied to clipboard

Marco Cognetta, Sangwhan Moon, Lawrence Wolf-sonkin, Naoaki Okazaki

HMoE: Heterogeneous Mixture of Experts for Language Modeling (2025.emnlp-main)

Challenge:	Character-level language modeling has been shown to perform well on highly agglutinative or morphologically rich languages while using only a small fraction of the parameters required by (sub)word models.
Approach:	They propose a “three-hot” embedding and decoding scheme that exploits the decomposability of Korean characters to model at the syllable level but using only jamo-level representations.
Outcome:	The proposed model reduces the embedding parameters by 99.6% and does not lose translation quality compared to the baseline model.

Copied to clipboard

An Wang, Xingwu Sun, Ruobing Xie, Shuaipeng Li, Jiaqi Zhu, Zhen Yang, Pinxue Zhao, Weidong Han, Zhanhui Kang, Di Wang, Naoaki Okazaki, Cheng-zhong Xu

Semantic Specialization for Knowledge-based Word Sense Disambiguation (2023.eacl-main)

Challenge:	Mixture of Experts (MoE) models use homogeneous experts with diverse capacities, resulting in a lack of expert specialization and parameter utilization.
Approach:	They propose a framework where experts differ in size and possess diverse capacities . they propose HMoE to encourage frequent activation of smaller experts .
Outcome:	The proposed framework outperforms homogeneous homogenous MoE models on evaluation benchmarks and achieves lower loss rate with fewer activated parameters.

Copied to clipboard

Sakae Mizuki, Naoaki Okazaki

Causal Reasoning through Two Cognition Layers for Improving Generalization in Visual Question Answering (2023.emnlp-main)

Challenge:	Existing methods for knowledge-based Word Sense Disambiguation (WSD) use only lexical knowledge to adapt contextualized embeddings.
Approach:	They propose a semantic specialization where contextualized embeddings are adapted to the WSD task using only lexical knowledge.
Outcome:	The proposed method outperforms previous studies that adapt contextualized embeddings while controlling deviations from the original embeddables.

Copied to clipboard

Trang Nguyen, Naoaki Okazaki

Generative Data Augmentation for Aspect Sentiment Quad Prediction (2023.starsem-1)

Challenge:	Existing attempts to generalize VQA focus on unimodal aspects, overlooking enhancements in multimodal aspects.
Approach:	They propose to decompose the responsibility of each stage into distinct experts and a cognition-enabled component (CC) they prioritize answer predictions governed by pathways involving both CCs while disregarding answers produced by either CC.
Outcome:	The proposed model improves multimodal predictions by emphasizing causal reasoning factors.

Copied to clipboard

An Wang, Junfeng Jiang, Youmi Ma, Ao Liu, Naoaki Okazaki

Sentiment Analysis

Challenge:	Existing approaches to analyze text contain rewrites and inconsistency between text and quads.
Approach:	They propose a new approach to analyze aspect terms, opinion terms, sentiment polarity in text . they augment quads and train a quads-to-text model to generate corresponding texts .
Outcome:	The proposed method outperforms existing methods and achieves state-of-the-art performance on two datasets.

How You Prompt Matters! Even Task-Oriented Constraints in Instructions Affect LLM-Generated Text Detection (2024.findings-emnlp)

Copied to clipboard

Ryuto Koike, Masahiro Kaneko, Naoaki Okazaki

From Interpretability to Performance: Optimizing Retrieval Heads for Long-Context Language Models (2026.findings-acl)

Challenge:	Recent studies have presented LLM-generated-text detectors with promising performance, but they do not cover such diverse instruction patterns when creating datasets for LLM detection.
Approach:	They propose to use task-oriented constraints that would naturally be included in an instruction and are not related to detection-evasion to create detectors with large variances in detection performance.
Outcome:	The proposed detectors have a large variance in detection performance on student essay writing with task-oriented constraints, and the standard deviation is significantly larger than that on texts generated by the constraint with such a constraint.

Copied to clipboard

Youmi Ma, Naoaki Okazaki

ExaGPT: Example-Based Machine-Generated Text Detection for Human Interpretability (2026.findings-acl)

Challenge:	Recent studies on mechanistic interpretability revealed that long-context factuality is closely related to a set of attention heads, retrieval heads.
Approach:	They propose a method that generates training signals by contrasting normal model outputs with those from an ablated variant.
Outcome:	The proposed method achieves significant improvements on LLMs with a sparse retrieval score distribution.

Copied to clipboard

Ryuto Koike, Masahiro Kaneko, Ayana Niwa, Preslav Nakov, Naoaki Okazaki

Explainability & Interpretability in NLP

Challenge:	Existing interpretable detectors are not aligned with the human decision-making process and fail to offer evidence that users easily understand.
Approach:	They propose an interpretable detection approach that checks whether a text is human-written or LLM-generated by checking whether it shares more similar spans with human-generated texts.
Outcome:	ExaGPT outperforms interpretable detectors by +37.0 points at a false positive rate of 1%.

Transformer-based Lexically Constrained Headline Generation (2021.emnlp-main)

Copied to clipboard

Kosuke Yamada, Yuta Hitomi, Hideaki Tamori, Ryohei Sasano, Naoaki Okazaki, Kentaro Inui, Koichi Takeda

Enhancing Machine Translation with Dependency-Aware Self-Attention (2020.acl-main)

Challenge:	Existing automatic headline generation methods cannot include a given phrase in the generated headline.
Approach:	They propose a Transformer-based method that guarantees to include a given phrase in a generated headline.
Outcome:	The proposed method achieves ROUGE scores comparable to previous methods with Japanese news corpus.

Copied to clipboard

Emanuele Bugliarello, Naoaki Okazaki

Machine Translation

Challenge:	Currently, most neural machine translation models rely on pairs of parallel sentences, assuming syntactic information is automatically learned by an attention mechanism.
Approach:	They propose a parameter-free, dependency-aware self-attention mechanism that integrates syntactic knowledge into a Transformer model and propose 'a parameter free approach' they also propose - a novel mechanism that improves translation quality for long sentences and in low-resource scenarios.
Outcome:	The proposed approach improves translation quality on English-German and English-Turkish translation tasks and in low-resource scenarios.

Improving Truthfulness of Headline Generation (2020.acl-main)

Copied to clipboard

Kazuki Matsumaru, Sho Takase, Naoaki Okazaki

Debiasing Isn’t Enough! – on the Effectiveness of Debiasing MLMs and Their Social Biases in Downstream Tasks (2022.coling-1)

Challenge:	Existing studies on abstractive summarization report ROUGE scores, but are concerned about the truthfulness of generated summaries.
Approach:	They propose to remove untruthful instances from supervision data to improve headline generation . they build a binary classifier that predicts an entailment relation between an article and its headline .
Outcome:	The proposed model improves on two popular datasets.

Copied to clipboard

Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki

Positional Encoding to Control Output Sequence Length (N19-1)

Challenge:	Existing measures for social bias evaluation are inadequate for MLMs to accurately evaluate the social biases in their systems.
Approach:	They propose task-agnostic intrinsic and task-specific extrinsic social bias evaluation measures for MLMs that use different methods to re-learn social biases during fine-tuning on downstream tasks.
Outcome:	The findings highlight the limitations of existing MLM bias evaluation measures and raise concerns on the deployment of MLMs in downstream applications using those measures.

Copied to clipboard

Sho Takase, Naoaki Okazaki

Word-level Perturbation Considering Word Length and Compositional Subwords (2022.findings-acl)

Challenge:	Neural encoder-decoder models have been successful in natural language generation tasks, but they must be limited to a specified length for abstractive summarization.
Approach:	They propose a sinusoidal positional extension to preserve the length constraint so that a neural encoder-decoder model can generate a text of any length even if the target length is unseen in training data.
Outcome:	The proposed method can generate a text of any length even if the target length is unseen in training data and improves ROUGE scores.

Copied to clipboard

Tatsuya Hiraoka, Sho Takase, Kei Uchiumi, Atsushi Keyaki, Naoaki Okazaki

Challenge:	Word replacement considering length and compositional word replacement are effective word-level perturbations.
Approach:	They propose two simple modifications for word-level perturbation: Word Replacement considering Length and Compositional Word Replacement.
Outcome:	The proposed methods improve word-level perturbation and classification performance.

Two Counterexamples to Tokenization and the Noiseless Channel (2024.lrec-main)

Copied to clipboard

Marco Cognetta, Vilém Zouhar, Sangwhan Moon, Naoaki Okazaki

ExtraPhrase: Efficient Data Augmentation for Abstractive Summarization (2022.naacl-srw)

Challenge:	Nevertheless, Rényi efficiency is not perfect and the metric is difficult to evaluate because training multiple tokenizers can be prohibitively expensive and takes days or weeks.
Approach:	They propose to use Rényi efficiency as an intrinsic mechanism to evaluate a tokenizer for NLP tasks without the expensive step of training multiple models with different tokenizers.
Outcome:	The proposed metric is better correlated to downstream model performance than a percentile frequency metric.

Copied to clipboard

Mengsay Loem, Sho Takase, Masahiro Kaneko, Naoaki Okazaki

Gender Bias in Meta-Embeddings (2022.findings-emnlp)

Challenge:	Recent studies indicated that neural methods are governed by the scaling law for the amount of training data.
Approach:	They propose a low-cost strategy to augment training data for abstractive summarization tasks by extracting summarized text and paraphrasing it.
Outcome:	The proposed strategy outperforms back-translation and self-training and is more cost-efficient when training data is small.

Copied to clipboard

Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki

Interpretability for Language Learners Using Example-Based Grammatical Error Correction (2022.acl-long)

Challenge:	Existing methods to develop meta-embeddings from source embeddings contain unfair gender-related biases, and how these influence the meta-bedding has not been studied yet.
Approach:	They propose to use multiple debiasing methods on a single source embedding to create a gender-based meta-embedding.
Outcome:	The proposed method amplifies gender biases compared to input source embeddings.

Copied to clipboard

Masahiro Kaneko, Sho Takase, Ayana Niwa, Naoaki Okazaki

Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach (2024.findings-eacl)

Challenge:	Existing neural-based GEC models mainly aim at improving accuracy, but their interpretability has not been explored.
Approach:	They propose an example-based method that generates corrections using retrieved examples.
Outcome:	The proposed method improves interpretability and supports language learners.

Copied to clipboard

Masahiro Kaneko, Graham Neubig, Naoaki Okazaki

Explainability & Interpretability in NLP

Challenge:	Existing systems that make predictions and ask questions are unable to have a mutual exchange of opinions.
Approach:	They propose to use a dataset and computational framework to allow systems to have beneficial discussions with humans, improving the accuracy by 25 points on a natural language inference task.
Outcome:	The proposed system improves accuracy by 25 points on a natural language inference task.

Evaluating Gender Bias of Pre-trained Language Models in Natural Language Inference by Considering All Labels (2024.lrec-main)

Copied to clipboard

Panatchakorn Anantaprayoon, Masahiro Kaneko, Naoaki Okazaki

Reducing Sequence Length by Predicting Edit Spans with Large Language Models (2023.emnlp-main)

Challenge:	Existing methods to evaluate gender bias in PLMs focus on one label out of three labels, such as neutral.
Approach:	They propose a bias evaluation method for PLMs that considers all the three labels of NLI task and then defines a measure based on the corresponding label output.
Outcome:	The proposed method can distinguish biased, incorrect inferences from non-biased incorrect infertility better than baseline, resulting in a more accurate bias evaluation.

Copied to clipboard

Masahiro Kaneko, Naoaki Okazaki

Comparing Intrinsic Gender Bias Evaluation Measures without using Human Annotated Examples (2023.eacl-main)

Challenge:	Large Language Models (LLMs) have demonstrated remarkable performance in various tasks and gained significant attention.
Approach:	They propose to predict edit spans for local sequence transduction tasks by predicting edit span with a position of the source text and corrected tokens.
Outcome:	The proposed method reduces the length of the target sequence and the computational cost for inference by as small as 21%.

Copied to clipboard

Masahiro Kaneko, Danushka Bollegala, Naoaki Okazaki

Jamo Pair Encoding: Subcharacter Representation-based Extreme Korean Vocabulary Compression for Efficient Subword Tokenization (2020.lrec-1)

Challenge:	Existing methods to evaluate gender biases in pre-trained language models have been limited by the cost and difficulties of recruiting human annotators.
Approach:	They propose a method to compare intrinsic gender bias evaluation measures without relying on human annotated examples.
Outcome:	The proposed method compares gender-based gender bias evaluation measures without human annotators without human input.

Copied to clipboard

Sangwhan Moon, Naoaki Okazaki

OpenKorPOS: Democratizing Korean Tokenization with Voting-Based Open Corpus Annotation (2022.lrec-1)

Challenge:	a vocabulary size problem is unsolved in multilingual pre-training tasks.
Approach:	They propose two algorithms that increase the elasticity of budget for building the vocabulary in Byte-Pair Encoding inspired tokenizers, and reduce the cost of supporting Korean in a multilingual model.
Outcome:	The proposed method reduces the cost of supporting Korean in a multilingual model.

Copied to clipboard

Sangwhan Moon, Won Ik Cho, Hye Joo Han, Naoaki Okazaki, Nam Soo Kim

Distributional Properties of Subword Regularization (2024.emnlp-main)

Challenge:	Korean uses spaces at larger-than-word boundaries, unlike other East-Asian languages.
Approach:	They propose to use Korean morphological analyzers to provide a sequence of morpheme-level tokens, losing information in the tokenization process.
Outcome:	The proposed scheme improves existing tagging scheme and makes it friendlier to generative tasks.

Copied to clipboard

Marco Cognetta, Vilém Zouhar, Naoaki Okazaki