Papers by Hitomi Yanaka
Does Character-level Information Always Improve DRS-based Semantic Parsing? (2023.starsem-1)
Copied to clipboard
| Challenge: | incorporating character-level information does not improve the performance in English and German, and is not sensitive to correct character order in Dutch. |
| Approach: | They propose to incorporate character-level representations into a neural semantic parser for Discourse Representation Structures and to test their performance using order of character sequences. |
| Outcome: | The proposed parser improves in English, German, Dutch, and Italian in four languages. |
On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing decoder-based pre-trained language models demonstrate excellent multilingual capabilities, but it is unclear how they handle multilingualism. |
| Approach: | They propose to examine the neuron-level internal behavior of decoder-based PLMs by finding neurons that fire “uniquely for each language” within decoded PLM models. |
| Outcome: | The proposed models fire “uniquely for each language” and show that language-specific neurons are unique, with a slight overlap (5%) between languages. |
SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models have limitations to generalize to diverse semantic phenomena, and it is unclear whether they can capture compositional meanings. |
| Approach: | They propose a systematic generalization testbed based on Natural language semantics to map natural language sentences to multiple meaning representations. |
| Outcome: | The proposed model can generalize to unseen combinations of quantifiers, negations, and modifiers, but not to the others. |
Exploring Intra and Inter-language Consistency in Embeddings with ICA (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that ICA can reveal universal semantic axes across languages but lack verification of consistency of independent components within and across languages. |
| Approach: | They propose to use independent component analysis to identify independent components that are more interpretable than PCA to find universal semantic axes. |
| Outcome: | The proposed framework ensures the reliability and universality of semantic axes. |
Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers (2026.findings-acl)
Copied to clipboard
Qingcheng Zeng, Yuheng Lu, Zeqi Zhou, Heli Qi, Puxuan Yu, Fuheng Zhao, Hitomi Yanaka, Weihao Xuan, Naoto Yokoya
| Challenge: | a new study examines the performance of code-switching IR in monolingual contexts . code-witching is a pervasive linguistic phenomenon in global communication . |
| Approach: | They propose a benchmark to evaluate code-switching IR in monolingual contexts . they propose CS-MTEB, which measures performance declines of up to 27% . |
| Outcome: | The proposed benchmark shows that code-switching performance is degraded by 27% . the proposed benchmark is based on a dataset of mixed-language queries . |
Analyzing Syntactic Generalization Capacity of Pre-trained Language Models on Japanese Honorific Conversion (2023.starsem-1)
Copied to clipboard
| Challenge: | Using Japanese honorifics requires knowledge of grammatical rules and contextual information, such as social relationships. |
| Approach: | They propose a Japanese honorific conversion task that considers social relationships among people mentioned in a conversation. |
| Outcome: | The proposed model performed better on the context-aware task than the prompt-based one. |
Acquisition of Phrase Correspondences Using Natural Deduction Proofs (N18-1)
Copied to clipboard
| Challenge: | Existing methods for Recognizing Textual Entailment (RTE) lack phrasal knowledge. |
| Approach: | They propose a method for detecting paraphrases via natural deduction proofs of semantic relations between sentence pairs. |
| Outcome: | The proposed method detects paraphrases that are absent from existing paraphrase databases and improves accuracy of RTE tasks. |
Multimodal Logical Inference System for Visual-Textual Entailment (P19-2)
Copied to clipboard
| Challenge: | Recent studies of multimodal inference provide challenging tasks such as visual question answering and visual reasoning. |
| Approach: | They propose an unsupervised multimodal logical inference system that can prove entailment relations between texts and images by combing semantic parsing and theorem proving. |
| Outcome: | The proposed system can handle semantically complex sentences for visual-textual inference. |
Constructing Multilingual Code Search Dataset Using Neural Machine Translation (2023.acl-srw)
Copied to clipboard
| Challenge: | Existing datasets for code search are monolingual, but their query data are only in English. |
| Approach: | They construct a multilingual code search dataset in four natural and four programming languages using a neural machine translation model and apply back-translation data filtering to it. |
| Outcome: | The proposed model pre-trained with all natural and programming language data performs best under almost all settings. |
Topic Modeling for Short Texts with Large Language Models (2024.acl-srw)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can be used to solve topic modeling challenges for short texts by contextually learning the meanings of words. |
| Approach: | They propose two approaches to using Large Language Models (LLMs) for topic modeling: parallel prompting and sequential prompting. |
| Outcome: | The proposed methods identify more coherent topics than existing ones while maintaining the diversity of the induced topics. |
Reforging : A Method for Constructing a Linguistically Valid Japanese CCG Treebank (2024.eacl-srw)
Copied to clipboard
| Challenge: | Existing treebanks for Combinatory Categorial Grammar (CCG) are insufficient for linguistic validity of CCG . |
| Approach: | They propose to combine ABCTreebank and lightblue to generate a linguistically valid Japanese CCG treebank with detailed information by filtering lightblu's lexical items using ABCTtreebank. |
| Outcome: | The proposed method generates a linguistically valid Japanese CCG treebank with detailed information by combining the strengths of ABCTreebank and lightblue. |
Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in language models have demonstrated their ability to process linguistic expressions with complex syntactic structures. |
| Approach: | They investigate whether language models employ shared neural mechanisms across different constructions by applying causal interpretability methods at a granular level. |
| Outcome: | The proposed model performance improves on acceptability judgment benchmarks. |
Do Neural Models Learn Systematicity of Monotonicity Inference in Natural Language? (2020.acl-main)
Copied to clipboard
| Challenge: | Despite the success of language models using neural networks, it remains unclear to what extent neural models have the generalization ability to perform inferences. |
| Approach: | They propose a method to evaluate whether neural models can learn systematicity of monotonicity inference in natural language. |
| Outcome: | The proposed method shows that neural models can perform inferences on unseen combinations of lexical and logical phenomena when syntactic structures are similar between training and test sets. |
NeuronMoE: Efficient Cross-Lingual Extension via Neuron-Guided Mixture-of-Experts (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches allocate experts based on layer-level similarity, yet language processing exhibits fine-grained specialization at individual neurons. |
| Approach: | They propose a method that analyzes language-specific neurons to guide expert allocation per layer based on cross-lingual neuron diversity. |
| Outcome: | The proposed method reduces the complexity of the model by 40% while matching the performance of the LayerMoE baseline. |
Logical Inference for Counting on Semi-structured Tables (2022.acl-srw)
Copied to clipboard
| Challenge: | Natural Language Inference (NLI) tasks require numerical understanding to perform a numerical type of inference, such as counting. |
| Approach: | They propose a logical inference system for reasoning between semi-structured tables and texts that uses logical representations as meaning representations and model checking to handle a numerical type of inference. |
| Outcome: | The proposed system can perform inference with numerical comparatives with tables and texts in English. |
Analysis of the Neglect-Zero Effect in Large Language Models (2026.acl-srw)
Copied to clipboard
| Challenge: | a neglect-zero effect is a human cognitive bias in language processing . it is unclear whether LLMs also exhibit this effect . |
| Approach: | They focus on a human cognitive bias called the *neglect-zero effect* . they propose a paradigm where exposure to a preceding sentence facilitates processing of a subsequent sentence due to their similarity. |
| Outcome: | The proposed paradigm based on priming facilitates processing of a subsequent sentence due to their similarity to the target. |
Compositional Semantics and Inference System for Temporal Order based on Japanese CCG (2022.acl-srw)
Copied to clipboard
| Challenge: | a system for temporal order in Japanese has not been developed for linguistic inference involving temporal expressions. |
| Approach: | They propose a Japanese NLI system that considers temporal order in Japanese . they use axioms for temporal relations and automated theorem provers to perform inference involving temporal orders. |
| Outcome: | The proposed system outperforms logic-based systems and current deep learning models on Japanese datasets. |
Action Inference for Destination Prediction in Vision-and-Language Navigation (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing work on vision-and-language navigation focuses on spatial reasoning and semantic grounding of visual information, but there is still scope for improvement. |
| Approach: | They propose a VLN task of destination prediction for picking up a pedestrian that requires action inference from a crowd-sourced dataset. |
| Outcome: | The proposed model can reason about the effect of the next action and the next on the destination to a certain extent. |
Visual-Textual Entailment with Quantities Using Model Checking and Knowledge Injection (2024.lrec-main)
Copied to clipboard
| Challenge: | Visual-textual entailment (VTE) is a critical task in multimodal inference. |
| Approach: | They propose a visual-textual entailment system that solves VTE tasks with quantities and negation. |
| Outcome: | The proposed system solves visual-textual entailment tasks with quantities and negation more robustly than previous approaches. |
Compositional Evaluation on Japanese Textual Entailment and Similarity (2022.tacl-1)
Copied to clipboard
| Challenge: | Despite growing interest in linguistic universals, most NLI/STS studies focus on English. |
| Approach: | They propose a Japanese NLI/STS dataset that was manually translated from the English dataset SICK. |
| Outcome: | The proposed datasets show that pre-trained language models are insensitive to word order and case particles. |
GesNavi: Gesture-guided Outdoor Vision-and-Language Navigation (2024.eacl-srw)
Copied to clipboard
| Challenge: | Existing datasets for outdoor Vision-and-Language Navigation (VLN) tasks do not include verbal instructions for communicating with mobility. |
| Approach: | They propose a dataset for gesture-guided outdoor VLN instructions with demonstrative expressions that incorporates gestures and linguistic commands. |
| Outcome: | The proposed datasets are compared against existing datasets and analysed in detail. |
Bias Mitigation or Cultural Commonsense? Evaluating LLMs with a Japanese Dataset (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that large language models exhibit social biases . however, debiasing methods may degrade the capabilities of LLMs if they are not properly evaluated . |
| Approach: | They propose a Japanese benchmark to evaluate social biases and cultural commonsense in large language models in a unified format. |
| Outcome: | The proposed method degrades the performance of the LLMs on the cultural commonsense task by 75%. |
Evaluation of Multilingual Ability to Use Spatial Deictic Expressions in Vision-Language Models (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing studies have focused on the ability of vision-language models to utilize spatial deictic expressions, which depend on the situation of utterance. |
| Approach: | They develop a benchmark to evaluate the multilingual ability of VLMs to use spatial deictic expressions in four languages. |
| Outcome: | The proposed models use demonstratives in a different manner from humans, particularly in selecting demonstrative based on distance from the object. |
What Do Vision–Language Models Encode for Personalized Image Aesthetics Assessment? (2026.findings-acl)
Copied to clipboard
| Challenge: | Personalized image aesthetics assessment (PIAA) is an important research problem with practical applications. |
| Approach: | They propose a vision-language model that encodes multi-level aesthetic attributes . they analyze visual representations of VLMs to examine their internal representations . |
| Outcome: | The proposed framework can be used to personalize images without fine-tuning . it can be implemented in a variety of image domains and architectures. |
Exploring Transitivity in Neural NLI Models through Veridicality (2021.eacl-main)
Copied to clipboard
| Challenge: | Despite recent success of deep neural networks in natural language processing, the extent to which they can demonstrate human-like generalization capacities remains unclear. |
| Approach: | They propose an analysis method to evaluate whether models can draw inferences composed of veridical inference and arbitrary inference types. |
| Outcome: | The proposed model performs poorly on transitivity inference tasks, suggesting it lacks generalization capacity for drawing composite inferences from training examples. |
Evaluating Structural Generalization in Neural Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have focused on compositional generalization with semantic parsing, but it remains unclear to what extent models can translate sentences that require structural generalization. |
| Approach: | They construct a machine translation dataset that measures compositional generalization with control of words and sentence structures. |
| Outcome: | The proposed model struggle more in structural generalization than in compositional generalization. |
Do Grammatical Error Correction Models Realize Grammatical Generalization? (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models for grammatical error correction use pseudo data, but they are inconvenient for realworld deployment due to large amounts of training data. |
| Approach: | They propose a method to evaluate whether GEC models can generalize to unseen errors by using synthetic and real GEC datasets with controlled vocabularies. |
| Outcome: | The proposed model fails to realize grammatical generalization even in simple settings with limited vocabulary and syntax, suggesting it lacks the generalization ability required to correct errors from provided training examples. |
Jamp: Controlled Japanese Temporal Inference Dataset for Evaluating Generalization Capacity of Language Models (2023.acl-srw)
Copied to clipboard
| Challenge: | Existing datasets and analyses focusing on English do not address the need for resources in other languages. |
| Approach: | They propose a Japanese NLI benchmark focused on temporal inference . they use a set of temporal patterns to generate diverse examples . |
| Outcome: | The proposed model can perform fine-grained analysis in Japanese and English. |
Analyzing the Inner Workings of Transformers in Compositional Generalization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on compositional generalization abilities of neural models have focused on benchmarks, but the results do not reflect the underlying competence of the model. |
| Approach: | They propose to find an existing subnetwork that contributes to the generalization performance and perform causal analyses on how the model utilizes syntactic features. |
| Outcome: | The proposed model relies on syntactic features but the subnetwork with better generalization performance relies mainly on a non-compositional algorithm . |