Papers by Kouta Nakayama
Co-Teaching Student-Model through Submission Results of Shared Task (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Shared tasks require participants to submit only system outputs and descriptions. |
| Approach: | They propose to utilize all system outputs in a shared task to build a unified system that performs better than the task's single best system. |
| Outcome: | The proposed scheme outperforms the best system in the SHINRA2019-JP shared task with nine participants. |
Are Checklists Really Useful for Automatic Evaluation of Generative Tasks? (2025.emnlp-main)
Copied to clipboard
| Challenge: | evaluators using large language models face ambiguous criteria and inconsistent evaluations. |
| Approach: | They investigate whether checklists should be used for all questions or selectively . they generate checklists using six methods and evaluate their effectiveness across eight models . |
| Outcome: | The proposed method improves evaluation performance in pairwise comparisons while ignoring human-written criteria. |
Resource of Wikipedias in 31 Languages Categorized into Fine-Grained Named Entities (2022.coling-1)
Copied to clipboard
| Challenge: | a resource of Wikipedias in 31 languages is categorized into Extended Named Entity (ENE) ENE version 8 has 219 fine-grained NE categories. |
| Approach: | They describe a resource of Wikipedias in 31 languages categorized into Extended Named Entity (ENE) they first categorized 920 K Japanese Wikipedia pages using machine learning, then shared a task of Wikipedia categorization into 30 languages . |
| Outcome: | The proposed system is based on a dataset of Japanese Wikipedia pages . the dataset shows the best performance among the 30 languages . |
Task Assignment meets Annotator Modeling: Human-LLM Collaborative Annotation with Constraints (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing approaches to label annotation are labor-intensive and time-consuming. |
| Approach: | They propose a framework that estimates per-task accuracy from task features using a learning from crowds model and incorporates these estimations into a linear programming formulation that assigns tasks under practical constraints. |
| Outcome: | The proposed method achieves comparable accuracy to baseline methods while satisfying given constraints. |
Constructing a Japanese Verdict Prediction Dataset for Fact-Checking of LLM-Generated Texts (2026.acl-srw)
Copied to clipboard
Miwa Masano, Hirokazu Kiyomaru, Atsushi Keyaki, Kaito Horio, Rei Minamoto, Ribeka Keyaki, Kouta Nakayama, Hideyuki Tachibana, Daisuke Kawahara
| Challenge: | Text generated by Large Language Models (LLMs) may contain plausible but incorrect information known as hallucinations. |
| Approach: | They extend the label set for verdict prediction to capture claim-evidence relationships humans would commonly interpret as supported or refuted. |
| Outcome: | The proposed system improves F1 by 4 percentage points compared to baseline. |