Papers by Hayate Iso
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization (2025.findings-naacl)
Copied to clipboard
| Challenge: | a recent study investigated hallucinations in multi-document summarization tasks . but, it is unclear how challenges arising from handling multiple documents affect outputs . |
| Approach: | They investigate how hallucinations manifest in large language models when summarizing topic-specific information from a set of documents. |
| Outcome: | The proposed benchmarks show that the models generate more hallucinations than baselines . the results highlight the need for more effective approaches to mitigate hallucinosity in MDS . |
Retrieval Helps or Hurts? A Deeper Dive into the Efficacy of Retrieval Augmentation to Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LMs) excel in retrieving popular facts, but encounter difficulty with infrequent entity-relation pairs compared to retrievers. |
| Approach: | They propose to use a WiTQA dataset to explore the effects of combinations of entities and relations on LMs. |
| Outcome: | The proposed model can retain popular relations of less common entities while retaining the same popular relations. |
XATU: A Fine-grained Instruction-based Benchmark for Explainable Text Updates (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing text editing benchmark datasets contain coarse-grained instructions and lack explainability, resulting in outputs that deviate from intended changes. |
| Approach: | They propose a benchmark specifically designed for fine-grained instruction-based explainable text editing. |
| Outcome: | The proposed benchmark incorporates fine-grained instructions and gold-standard edit explanations. |
Learning to Select, Track, and Generate for Data-to-Text (P19-1)
Copied to clipboard
Hayate Iso, Yui Uehara, Tatsuya Ishigaki, Hiroshi Noji, Eiji Aramaki, Ichiro Kobayashi, Yusuke Miyao, Naoaki Okazaki, Hiroya Takamura
| Challenge: | Existing models often refer to the same data record multiple times. |
| Approach: | They propose a data-to-text generation model with two modules, one for tracking and the other for text generation. |
| Outcome: | The proposed model outperforms existing models even without writer information in all evaluation metrics and contributes to content planning and surface realization. |
AmbigNLG: Addressing Task Ambiguity in Instruction for NLG (2024.emnlp-main)
Copied to clipboard
| Challenge: | AmbigNLG is a novel task designed to tackle task ambiguity in instructions for NLG . ambiguous instructions often impede the performance of Large Language Models (LLMs) . |
| Approach: | They propose an ambiguity taxonomy that categorizes different types of instruction ambiguities and refines initial instructions with clearer specifications. |
| Outcome: | The proposed task improves alignment of generated text with user expectations, achieving 15.02-point increase in ROUGE scores. |
Convex Aggregation for Opinion Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in text autoencoders have significantly improved the quality of the latent space, allowing models to generate consistent text from aggregated latent vectors. |
| Approach: | They develop a framework which searches input-output word overlap for latent vector aggregation. |
| Outcome: | The proposed framework improves the quality of the latent space and establishes state-of-the-art performance on two opinion summarization benchmarks. |
Less is More for Long Document Summary Evaluation by LLMs (2024.eacl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown promising performance in summary evaluation tasks, yet they face challenges such as high computational cost and the Lost-in-the-middle problem where important information in the middle of long documents is often overlooked. |
| Approach: | They propose a novel method which extracts key sentences from a long source document and then evaluates the summary by prompting LLMs. |
| Outcome: | The proposed method significantly reduces evaluation costs and exhibits a higher correlation with human evaluations. |
Fact-based Text Editing (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for fact-based text editing are not suitable for all situations. |
| Approach: | They propose a method for automatically generating a dataset with a draft text, revised text, and several facts represented in triples. |
| Outcome: | The proposed method outperforms the encoder-decoder approach on two datasets and shows that it conducts inference faster than the encoded-decoding approach. |
Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education (2025.naacl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have potential to automate hiring but inherent biases may lead to unfair hiring practices. |
| Approach: | They evaluate how factors such as gender, race, and educational background influence model decisions. |
| Outcome: | The proposed model reduces biases related to gender and race, but implicit biase concerning educational background remains significant. |
Comparative Opinion Summarization via Collaborative Decoding (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing opinion summarization methods are insufficient to help users compare multiple choices. |
| Approach: | They propose a comparative opinion summarization task that generates two contrastive summaries and one common summary from two different candidate sets of reviews. |
| Outcome: | The proposed framework produces higher-quality contrastive and common summaries than state-of-the-art models. |