Papers by Yusuke Iwasawa
Dynamic Injection of Entity Knowledge into Dense Retrievers (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Dense retrievers struggle with queries involving less-frequent entities due to limited entity knowledge. |
| Approach: | They propose a BERT-based retriever enhanced with a context-entity attention layer and dynamically updatable entity embeddings. |
| Outcome: | The proposed retriever incorporates external entity knowledge without retraining. |
Which Programming Language and What Features at Pre-training Stage Affect Downstream Logical Inference Performance? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have demonstrated remarkable generalization abilities in mathematics and reasoning tasks. |
| Approach: | They pre-trained decoder-based language models from scratch using ten programming languages and three natural language datasets. |
| Outcome: | The proposed models outperform natural languages on logical reasoning tasks. |
On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing decoder-based pre-trained language models demonstrate excellent multilingual capabilities, but it is unclear how they handle multilingualism. |
| Approach: | They propose to examine the neuron-level internal behavior of decoder-based PLMs by finding neurons that fire “uniquely for each language” within decoded PLM models. |
| Outcome: | The proposed models fire “uniquely for each language” and show that language-specific neurons are unique, with a slight overlap (5%) between languages. |
Investigating the Multilingual Calibration Effects of Language Model Instruction Tuning (2026.eacl-short)
Copied to clipboard
Jerry Huang, Peng Lu, Qiuhao Zeng, Yusuke Iwasawa, Yutaka Matsuo, Sarath Chandar, Edison Marrese-Taylor, Irene Li
| Challenge: | despite advances in foundation model research, the relationship between large language models and their calibration remains an open area of research. |
| Approach: | They examine a gap in the calibration of large language models within multilingual settings to better understand how data scarcity can potentially lead to different calibration effects. |
| Outcome: | The proposed calibration gap is found in two multilingual benchmarks over 29 and 42 languages. |
Slender-Mamba: Fully Quantized Mamba in 1.58 Bits From Head to Toe (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved significant performance improvements in natural language processing domain, but require large computational resources for training and inference. |
| Approach: | They propose to use a language model architecture based on State-Space Models to quantify embedding and projection layers of a model with 150 B tokens from scratch. |
| Outcome: | The proposed language model architecture reduces costs by compressing context windows during inference while reducing the cost of training and inference. |
Language Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy Arithmetic Tasks (2024.acl-short)
Copied to clipboard
| Challenge: | Despite the generality and far-reaching consequences of large language models, there are still significant limitations making it difficult to apply them to certain tasks. |
| Approach: | They show that large language models can perform arithmetic tasks more robustly when conditioned on all of the correct higher-order digits. |
| Outcome: | The proposed model can predict the first digit of n-digit by m-digit multiplication without chain of thought reasoning, but in practice it fails to correctly predict the last digit on n digit by 1-digit multiplikation . |
Inconsistent Tokenizations Cause Language Models to be Perplexed by Japanese Grammar (2025.acl-short)
Copied to clipboard
| Challenge: | Standard benchmarks for language models fail to capture nuanced capabilities such as the ability of language models to recognize and obey rare grammar points. |
| Approach: | They find that Weblab's uniformly bad tokenization is a possible root cause for its good performance . |
| Outcome: | The proposed model consistently assigns higher perplexity to ungrammatical psych predicate sentences than grammaticals. |
Lost in the Distance: Large Language Models Struggle to Capture Long-Distance Relational Knowledge (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent large language models have demonstrated impressive capabilities in handling long contexts . however, as context length increases, LLMs struggle more with filtering out irrelevant information . |
| Approach: | They propose to use unrelated sentences to capture relational knowledge over long contexts . they find that LLMs can handle edge noise with little impact, but can reason about distant relationships . |
| Outcome: | The proposed model can handle edge noise with little impact, but its ability to reason about distant relationships declines as the noise grows. |
When Instructions Multiply: Measuring and Estimating LLM Capabilities of Multiple Instructions Following (2025.findings-emnlp)
Copied to clipboard
Keno Harada, Yudai Yamazaki, Masachika Taniguchi, Edison Marrese-Taylor, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo
| Challenge: | a large number of languages are increasingly used to evaluate their ability to follow multiple instructions simultaneously. |
| Approach: | They propose two benchmarks to evaluate LLMs' ability to follow multiple instructions simultaneously . they use many instruction-following eval and style-aware Mostly Basic programming problems . |
| Outcome: | The proposed models predict performance on unseen instruction combinations and not used during training with 10% error. |
Answer When Needed, Forget When Not: Language Models Pretend to Forget via In-Context Knowledge Unlearning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly required to selectively unlearn specific information. |
| Approach: | They propose a method which fine-tunes pre-trained LLMs to enable prompt unlearning of target knowledge within the context while preserving unrelated information. |
| Outcome: | The proposed method achieves up to 95% forget accuracy while retaining 80% of unrelated knowledge, significantly outperforming baselines in both in-domain and out-of-domain scenarios. |
Revealing Redundant Syntax in Large Language Models through Multi-Hop Dependency Paths (2026.findings-eacl)
Copied to clipboard
| Challenge: | Prior work on attention–syntax alignment has focused on single-hop Universal Dependency edges (DPs). |
| Approach: | They extract 2–3 hop MDPs from UD-parsed English and quantify head–relation alignment with an Unlabeled Attachment Score (UAS)-style metric modified for causal masking in decoder-only models. |
| Outcome: | The authors show that head alignments are overlapped and specialized . the head alignment is measurable in large language models trained on raw text . |
Making Use of Latent Space in Language GANs for Generating Diverse Text without Pre-training (2021.eacl-srw)
Copied to clipboard
| Challenge: | Existing models for generating diverse texts are not pre-trained . generative adversarial networks suffer from mode-collapsing if they are not trained . |
| Approach: | They propose a GAN model that produces diverse texts conditioned by latent code . they propose to use Gumbel-Softmax distribution for word sampling . |
| Outcome: | The proposed model is competitive with existing models, which requires pre-training. |
Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for uncertainty quantification in large language models rely on indirect signals, such as entropy across sampled generations, which can be difficult to interpret and do not fully leverage the model’s ability to assess its own uncertainty. |
| Approach: | They propose a method that groups sampled generations into semantically distinct clusters and uses the probability assigned by the LLM to each option as a confidence estimate. |
| Outcome: | The proposed method outperforms baseline methods and achieves competitive performance with as few as two additional samples. |
ReAgent: Reversible Multi-Agent Reasoning for Knowledge-Enhanced Multi-Hop QA (2025.emnlp-main)
Copied to clipboard
Zhao Xinjie, Fan Gao, Xingyu Song, Yingjian Chen, Rui Yang, Yanran Fu, Yuyang Wang, Yusuke Iwasawa, Yutaka Matsuo, Irene Li
| Challenge: | Multi-hop question answering (QA) is a central challenge in natural language processing . early mistakes can cause errors and undermine the final result, authors say . |
| Approach: | They propose a reversible multi-agent reasoning framework that backtracks to earlier valid states when conflicts arise. |
| Outcome: | Empirical evaluation shows that the framework improves on forward-only benchmarks by 6% . the approach enables agents to backtrack to valid states when conflicts arise . |
MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)
Copied to clipboard
Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li
| Challenge: | Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities. |
| Approach: | They propose a comprehensive benchmark covering 29 languages, built on an English benchmark. |
| Outcome: | The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark. |
Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models (2026.eacl-short)
Copied to clipboard
| Challenge: | Large language models have limited truthfulness and tendency toward overconfidence constrain reliability in factual tasks. |
| Approach: | They propose an efficient method that leverages semantic information encoded in LLMs to quantify uncertainty. |
| Outcome: | The proposed method achieves comparable performance to baselines while significantly reducing computational overhead. |
Infinity-MoE: Generalizing Mixture of Experts to Infinite Experts (2026.eacl-short)
Copied to clipboard
| Challenge: | Existing methods to increase the number of experts are -MoE and . |
| Approach: | They propose a mixture of experts that selects a few feed-forward networks per token to increase the number of experts. |
| Outcome: | The proposed model improves on a GPT-2 Small model with 129M active and 186M total parameters by 2.5% over the current model. |
Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models exhibit remarkable proficiency in many tasks, but much about their inner workings remains unclear. |
| Approach: | They propose a test suite to measure the resilience of Large Language Models (LLMs) when subjected to extensive character-level permutations. |
| Outcome: | The proposed test suite measures the resilience of large language models to scrambled inputs. |
ClinDet-Bench: Beyond Abstention, Evaluating Judgment Determinability of LLMs in Clinical Decision-Making (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing medical benchmarks are insufficient to evaluate LLMs under incomplete information. |
| Approach: | They develop a benchmark that decomposes incomplete information into determinable and undeterminable conditions. |
| Outcome: | a clinical scoring system can decompose incomplete information into determinable and undeterminable conditions. a new benchmark, ClinDet-Bench, can evaluate determinability under incomplete information. |
Understanding Emergent Misalignment via Feature Superposition Geometry (2026.acl-long)
Copied to clipboard
| Challenge: | Emergent misalignment is a problem for large language models (LLMs) fine-tuning on narrow tasks can induce harmful behaviors despite no explicit supervision. |
| Approach: | They propose a mechanistic account based on the geometry of feature superposition . they propose to use sparse autoencoders to identify misalignment-inducing features . |
| Outcome: | The proposed model outperforms random removal and stronger mitigations than LLM-as-a-judge filtering. |