Papers by Koh Takeuchi
AHP-Powered LLM Reasoning for Multi-Criteria Evaluation of Open-Ended Responses (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Question answering (QA) tasks have been extensively studied in the field of natural language processing. |
| Approach: | They propose a method that leverages large language models and the analytic hierarchy process to assess open-ended questions. |
| Outcome: | The proposed method more closely aligns with human judgment compared to baselines on four datasets. |
Evaluating Saliency Explanations in NLP by Crowdsourcing (2024.lrec-main)
Copied to clipboard
| Challenge: | a crowdsourced method to evaluate saliency methods in NLP is proposed . saliencies are difficult for humans to understand, and can cause psychological harm . |
| Approach: | They propose a method to evaluate saliency methods in NLP by crowdsourcing . they recruited 800 crowd workers and empirically evaluated seven salience methods . |
| Outcome: | The proposed method evaluates saliency methods on two datasets using crowdsourced data . it shows that the results are comparable to existing methods on NLP and CV fields . |
MAPLE: Multi-Aspect Panels of LLM Evaluators for Open-Ended Questions (2026.findings-acl)
Copied to clipboard
| Challenge: | LLM-as-a-Judge uses LLMs to evaluate open-ended questions . however, the discrepancy between LLM generated evaluations and human evaluations remains a critical problem in this field . |
| Approach: | They propose a framework that orchestrates evaluations across multiple criteria using multiple LLMs. |
| Outcome: | The proposed framework achieves superior alignment with human evaluations compared to baselines. |
A Speculative and Tentative Common Ground Handling for Efficient Composition of Uncertain Dialogue (2022.lrec-1)
Copied to clipboard
| Challenge: | a study explores how the grounding process is composed and adapts to human cognitive processes . common ground is a set of information shared among participants that serves as a precondition for understanding individual utterances . |
| Approach: | a study investigates how the grounding process is composed by participants . it suggests that common ground may not necessarily be formed bottom-up through analytic expressions . |
| Outcome: | a new approach to human-like dialogue may be more suitable for natural human communication, the authors say . they show that common ground is mutually accepted among participants through holistic expressions . |