Papers by Masafumi Oyamada
LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to optimize large language models rely on manual design or focus on optimizing individual components. |
| Approach: | They propose a LaMDAgent framework that constructs and optimizes end-to-end post-training pipelines by exploring various model improving methods, objects, and their applied orderings based on task-based feedback. |
| Outcome: | The proposed framework achieves a 9.0-point gain in tool-use accuracy without degrading instruction-following, and reduces computational costs. |
On Synthesizing Data for Context Attribution in Question Answering (2025.acl-long)
Copied to clipboard
Gorjan Radevski, Kiril Gashteovski, Shahbaz Syed, Christopher Malon, Sebastien Nicolas, Chia-Chien Hung, Timo Sztyler, Verena Heußer, Wiem Ben Rim, Masafumi Enomoto, Kunihiro Takeoka, Masafumi Oyamada, Goran Glavaš, Carolin Lawrence
| Challenge: | Large Language Models (LLMs) have a tendency to hallucinate, resulting in false or misleading answers. |
| Approach: | They propose a novel generative strategy for synthesizing context attribution data. |
| Outcome: | The proposed approach is highly effective for fine-tuning small LMs for context attribution in different QA tasks and domains. |
Can Large Language Models Invent Algorithms to Improve Themselves? (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable performance improvements, but the methods for improving LLMs are still designed by humans. |
| Approach: | They propose a framework which enables LLMs to generate and learn model-improvement algorithms by the seed model. |
| Outcome: | The proposed framework outperforms human-designed methods in model-improving tasks and improves the seed model by 6% and outperformed human-design methods by 4.3% on GSM8k. |
SCAN: Semantic Document Layout Analysis for Textual and Visual Retrieval-Augmented Generation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent research indicates that using VLMs yields better RAG performance, but processing rich documents remains a challenge. |
| Approach: | They propose a VLM-friendly approach that enhances both textual and visual RAG systems. |
| Outcome: | The proposed approach outperforms conventional methods and commercial document processing solutions. |
Relevance, Diversity, and Exclusivity: Designing Keyword-augmentation Strategy for Zero-shot Classifiers (2024.starsem-1)
Copied to clipboard
| Challenge: | Existing methods incorporate semantically similar keywords related to class names, but the properties of effective keywords remain unclear. |
| Approach: | They propose a method for acquiring keywords that satisfy these properties without additional knowledge bases or data. |
| Outcome: | The proposed method outperforms existing methods in fully zero-shot and generalized zero- shot settings. |
Jellyfish: Instruction-Tuning Local Large Language Models for Data Preprocessing (2024.emnlp-main)
Copied to clipboard
| Challenge: | Until 2021, most efforts were concentrated on one or two specific tasks such as error detection (ED) and data imputation (DI). |
| Approach: | They propose to instruction tune local LLMs as universal DP task solvers that operate on a local, single, and low-priced GPU, ensuring data security and enabling further customization. |
| Outcome: | The proposed models deliver competitiveness and generalizability to unseen tasks while barely compromising the base models’ abilities in NLP tasks. |
Low-resource Taxonomy Enrichment with Pretrained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Taxonomies represent hierarchical relationships between terms or entities. |
| Approach: | They propose a framework for taxonomy enrichment in low-resource settings with pretrained language models as knowledge bases to compensate for the shortage of information. |
| Outcome: | The proposed framework predicts whether inputted term pairs have hierarchical relationships and leverages implicit knowledge from the LM to generate queries efficiently. |
Evaluating the Impact of Reviewer Guideline Design on LLM-Based Automated Peer Review (2026.findings-acl)
Copied to clipboard
| Challenge: | a growing workload has made peer review automation an urgent necessity, says a new study . official conference guidelines and reviewer-imitating guidelines degraded review performance . current human-based peer review system faces serious challenges, authors say . |
| Approach: | They analyze how reviewer guidelines influence automated peer review . official conference guidelines produce review results consistent with human judgments . |
| Outcome: | The proposed reviewer guidelines produce results consistent with human judgments . the proposed reviewers' imitations degraded performance, the authors note . |
Context Quality Matters in Training Fusion-in-Decoder for Extractive Open-Domain Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown that the quantity and quality of context affect retrieval-augmented generation models during training. |
| Approach: | They propose a method to mitigate overfitting to specific context quality by introducing bias to the cross-attention distribution. |
| Outcome: | The proposed method improves retrieval-augmented generation models on different context quality. |