Papers by Yoshi Suhara
Summarizing Community-based Question-Answer Pairs (2022.emnlp-main)
Copied to clipboard
| Challenge: | Community-based question answering (CQA) has become an essential component of online services. |
| Approach: | They propose a novel task to summarize CQA pairs into a concise summary . they use a benchmark dataset and a sentence-type transfer and deduplication removal approach . |
| Outcome: | The proposed task aims to create a concise summary from CQA pairs . the proposed method is stronger than existing methods and is publicly available . |
When2Call: When (not) to Call Tools (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on the accuracy of tool calling and less on when LMs should (not) call tools. |
| Approach: | They develop a benchmark which evaluates tool-calling decision-making . they leverage multiple-choice nature of the benchmark to develop 'preference optimization' training regime . |
| Outcome: | The proposed benchmark shows that state-of-the-art LMs show room for improvement on When2Call. |
Characterizing the Confidence of Large Language Model-Based Automatic Evaluation Metrics (2024.eacl-short)
Copied to clipboard
| Challenge: | Recent studies have focused on using Large Language Models (LLMs) to evaluate NLP tasks automatically. |
| Approach: | They characterize LLM evaluators’ confidence in ranking candidate NLP models and develop a configurable Monte Carlo simulation method to compensate for loss of correlation. |
| Outcome: | The proposed method can reach 95% confidence rankings of candidate models with reasonable evaluation set sizes. |
Source Identification in Abstractive Summarization (2024.eacl-short)
Copied to clipboard
| Challenge: | Existing studies define input sentences that contain essential information in the generated summary as source sentences. |
| Approach: | They define input sentences that contain essential information in the generated summary as source sentences and analyze the source sentences to determine how abstractive summaries are made. |
| Outcome: | The proposed method performs well in abstractive settings, while similarity-based methods perform robustly in extractive settings. |